ZeroMQ in details

In GIGAFLOW

In Gigaflow, this is “relatively” simple. Client should always be 1 and server should always be 2.

We only send a single port (which should hopefully be the correct server port) when sending our data to apex.

For finding this “server port” we:

1. Use a statistical algorithm to determine which port is used the most by src and dst ips, if its above a predefined threshold then we use this as the server port and can use that to determine 1 and 2.
2. If we don’t get above the threshold, we then see which side is using a port < 1024 and use this information to determine 1 and 2
3. If no port <1024 then we check our pre-defined application ports to determine 1 and 2
4. If no pre-defined match is found we then just treat the server(2) as the destination ip

With this, to apex we then send

1IP          2IP           ServerPort                         Bytes1->2            Bytes2->1            Packets1->2        Packets2->1        etc……

APEX  Packet

Before we get into Client vs Server there are a couple of things to understand regarding different types of analysis performed by Network Trending in Observer which is responsible for metadata creation in Apex.

 Types of trending in Observer:

1. Pairs Trending
   1. Creates top talker data on all traffic (with some exceptions for all)
      1. TopN MAC, IP, Application, VLAN, etc
   2. Uses Key Fields 1 -> 2, 1 <- 2
      1. Generally 1 = Client, 2 = Server
      2. Please see ‘Who is the client in Apex’ below for explanations
   3. Does not perform any response time analysis
2. Application Performance Analysis (APA)
   1. Performs response time analysis on all IP traffic
   2. Uses Key Fields Client IP and Server IP
3. Application Transaction Analysis (ATA)
   1. Performs analysis on transactions for specific applications
      1. Non standard port numbers running a known application will not be tracked
         1. Example: TCP 8080 is HTTP, but has not been defined in Observer
   2. Uses Key Fields Client IP and Server IP

Once the Client and Server IPs are understood, all other metrics are put into their corresponding Client/Server, 1 -> 2, and 1 <- 2 buckets.

Who is the client in Apex?

 TCP Based Conversations

1. If Observer sees a SYN packet
   1. The source is the Client, the destination is the Server
      1. Customers will sometimes complain that we have labeled a ‘Server’ as a Client
      2. They don’t initially realize that their server acted as a client to a backend Server
   2. If the port associated with the Server is defined, Observer will classify it as that defined application (TCP 80 = HTTP)
   3. If the port associated with the Server is not defined, Observer will classify it as Other
2. If Observer does not see a SYN packet, but either of the source or destination TCP Ports are a known port/application
   1. The IP Address associated with the known port is the server
      1. Example:
         1. Observer starts collecting traffic between IP 1/TCP Port 5987 and IP 2/TCP Port 80 but the beginning of the conversation is missing
         2. Because TCP Port 80 is HTTP & is defined in Observer, IP 2 will be the Server running HTTP
3. If Observer does not see a SYN packet and neither of the source or destination TCP ports are known
   1. Observer uses a hierarchy to try and determine what may be the Client versus Server
      1. Lowest port number
      2. First seen packet, Client is sender, Server is Receiver

UDP Based Conversations

1. UDP port number is defined
   1. The IP Addresses associated with the defined UDP port number is the server
2. UDP port number is not defined
   1. Observer uses a hierarchy to try to determine what may be the Client vs the Server

                                                               i.      Lowest port number

                                                             ii.      First seen packet, Client is sender, Server is Receiver

What happens in Trending when Observer sees traffic with ports that are not defined?

1. Pairs Trending
   1. Using Observer hierarchy as defined above, Pairs will put Client and Server metrics into specific Key Fields

                                                               i.      Client = 1 -> 2

                                                             ii.      Server = 1 <- 2

2. This includes all 1 -> 2, 1 <- 2 fields, including IP Address
3. Application Performance Analysis
   1. Using Observer hierarchy as defined above, APA will track metrics on the ‘Other’ application
   2. Observer will use all key fields surrounding APA with the exception of

                                                               i.      Client IP

                                                             ii.      Server IP

3. This will result in stats showing up for things like response time or network delay, but with no IP Addresses
4. Application Transaction Analysis
   1. No stats are collected for ATA

How does Apex deal with metadata from GigaFlow?