5G Uplink Classifier Using Loxilb

As we see more 5G Core CNFs adhering to native cloudnative principles I was immensely surprised when I stumbled upon Loxilb open source project because of the 5G related features that it supports out of the box.

The feature that really picked my interest is the UL-CL support. Basically UL-CL (uplink classifier) is used to divert specific UE traffic to local datacenters, this is especially useful in cases whereby the UPF does not support UL-CL and you want to route traffic destined to specific destination IP address to a local server. So Loxilb will actually perform two functions in this regard:

• Process GTP-U traffic that is passing through it and re-routing the traffic that meets the UL-CL policy.
• Create an load-balancer endpoint for the backend servers that will be receiving the UL-CL traffic.

In essence Loxilb is processing the UL-CL GTP-U and also serving as the loadbalancer for the backend servers that the subscribers are trying to access with respect to the UL-CL policy.

Also Loxilb is using eBPF for accelerating the traffic processing, eBPF has showing very good performance benefits compared to the normal kernel based traffic processing.

Fun fact: Loxilb can also be used an SCTP loadbalancer. It can even be used as an external loadbalancer solution for your Kubernetes cluster especially in situations where exotic protocol support is needed. A good use case for this will be when you have multiple AMFs and you need to load balance them with your gNBs.

Below is the architecture I used to try this out

LoxiLB UL-CL using UERANSIM and Open5gs

Sample commands:

./loxicmd create session user1 10.45.0.2 --accessNetworkTunnel=1:192.168.1.125 --coreNetworkTunnel=2:192.168.70.2

./loxicmd create sessionulcl user1 --ulclArgs=1:88.88.88.88

./loxicmd create lb 88.88.88.88 --tcp=2020:80 --endpoints=192.168.70.5:1

So in essence what the above is doing is to create a GTPU profile user1 first with the TEIDs (tunnel ID) i.e. the UE that has IP 10.45.0.2 and that is been served by gNB with IP 192.168.1.125 & TEID value of 1 and a UPF with IP 192.168.70.2 & TEID 2. This is used to properly de-encapsulate and encapsulate UL-CL traffic (in fact the gNB will not detect it is LoxiLB that is responding to the UL-CL GTP-U traffic request).

The user profile/session created in the above step will now be used to create the UL-CL configuration which means that the UE traffic with IP 10.45.0.2 that is encapsulated with the above GTP parameters and wants to access 88.88.88.88:2020 will be sent to the backend server 192.168.70.5:80.

Packet capture below shows some of the traffic that is captured at the web-server:

root@nginx-deployment-5d4fc6576c-dmdpw:/# tcpdump -i any host 10.45.0.2 -s 0 -nv
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
20:50:44.002227 IP (tos 0x0, ttl 63, id 13299, offset 0, flags [DF], proto TCP (6), length 60)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [S], cksum 0x1aaa (correct), seq 3730065449, win 65280, options [mss 1360,sackOK,TS val 601366506 ecr 0,nop,wscale 7], length 0
20:50:44.003761 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    192.168.70.5.80 > 10.45.0.2.45900: Flags [S.], cksum 0x110b (incorrect -> 0xd355), seq 901514344, ack 3730065450, win 65160, options [mss 1460,sackOK,TS val 2869781029 ecr 601366506,nop,wscale 7], length 0
20:50:44.011816 IP (tos 0x0, ttl 63, id 13300, offset 0, flags [DF], proto TCP (6), length 52)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [.], cksum 0xfea2 (correct), ack 1, win 510, options [nop,nop,TS val 601366516 ecr 2869781029], length 0
20:50:44.011831 IP (tos 0x0, ttl 63, id 13301, offset 0, flags [DF], proto TCP (6), length 132)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [P.], cksum 0x4a95 (correct), seq 1:81, ack 1, win 510, options [nop,nop,TS val 601366518 ecr 2869781029], length 80: HTTP, length: 80
        GET / HTTP/1.1
        Host: 88.88.88.88:2020
        User-Agent: curl/7.81.0
        Accept: */*
20:50:44.011948 IP (tos 0x0, ttl 64, id 54607, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.70.5.80 > 10.45.0.2.45900: Flags [.], cksum 0x1103 (incorrect -> 0xfe49), ack 81, win 509, options [nop,nop,TS val 2869781037 ecr 601366518], length 0
20:50:44.034886 IP (tos 0x0, ttl 64, id 54608, offset 0, flags [DF], proto TCP (6), length 290)
    192.168.70.5.80 > 10.45.0.2.45900: Flags [P.], cksum 0x11f1 (incorrect -> 0x6fc5), seq 1:239, ack 81, win 509, options [nop,nop,TS val 2869781060 ecr 601366518], length 238: HTTP, length: 238
        HTTP/1.1 200 OK
        Server: nginx/1.14.2
        Date: Mon, 10 Oct 2022 20:50:44 GMT
        Content-Type: text/html
        Content-Length: 612
        Last-Modified: Tue, 04 Dec 2018 14:44:49 GMT
        Connection: keep-alive
        ETag: "5c0692e1-264"
        Accept-Ranges: bytes
20:50:44.049242 IP (tos 0x0, ttl 63, id 13302, offset 0, flags [DF], proto TCP (6), length 52)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [.], cksum 0xfd25 (correct), ack 239, win 509, options [nop,nop,TS val 601366549 ecr 2869781060], length 0
20:50:44.049501 IP (tos 0x0, ttl 64, id 54609, offset 0, flags [DF], proto TCP (6), length 664)
    192.168.70.5.80 > 10.45.0.2.45900: Flags [P.], cksum 0x1367 (incorrect -> 0x3edf), seq 239:851, ack 81, win 509, options [nop,nop,TS val 2869781075 ecr 601366549], length 612: HTTP
20:50:44.056009 IP (tos 0x0, ttl 63, id 13303, offset 0, flags [DF], proto TCP (6), length 52)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [.], cksum 0xfaa8 (correct), ack 851, win 505, options [nop,nop,TS val 601366563 ecr 2869781075], length 0
20:50:44.059248 IP (tos 0x0, ttl 63, id 13304, offset 0, flags [DF], proto TCP (6), length 52)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [F.], cksum 0xfaa2 (correct), seq 81, ack 851, win 505, options [nop,nop,TS val 601366568 ecr 2869781075], length 0
20:50:44.059753 IP (tos 0x0, ttl 64, id 54610, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.70.5.80 > 10.45.0.2.45900: Flags [F.], cksum 0x1103 (incorrect -> 0xfa93), seq 851, ack 82, win 509, options [nop,nop,TS val 2869781085 ecr 601366568], length 0
20:50:44.067965 IP (tos 0x0, ttl 63, id 13305, offset 0, flags [DF], proto TCP (6), length 52)
    10.45.0.2.45900 > 192.168.70.5.80: Flags [.], cksum 0xfa92 (correct), ack 852, win 505, options [nop,nop,TS val 601366573 ecr 2869781085], length 0

Some few points to note:

• There is need to know about the TEIDs that the gNB and UPF will be using, this can be further done by finding a way to add some logic in the SMF to trigger an API call to LoxiLB (LoxiLB provides an API endpoint out of the box). Also an entry has to be added for each UE IP for now though LoxiLB engineers says they will try to have some logic that can be used to capture a group of UEs, this will further simplify/minimize the configuration that is needed.
• Since UL-CL traffic is been re-routed to a local server then it means that the compliance factor has to be considered especially if there are security controls that already exists in the 5G core (after the UPF has processed the traffic)

Also as part of the upcoming feature is the High Availability for the loxiLB so that traffic processing is not impacted when one of the instances goes down.

I hope this excites you just as it did for me.

I will like to thank the LoxiLB engineers for responding quickly to the issues I encountered and providing necessary code updates to get this working.

References

LoxiLB | Cloud-Native Load-balancer

loxilbdocs/gtp.md at main · loxilb-io/loxilbdocs

GitHub - loxilb-io/loxilb: loxilb provides service type external load-balancer for K8s using eBPF…