The 5G system is proposed to support a diverse set of vertical services with different QoS requirements. It is designed as a set of network functions (NFs) and follows CUPS to separate its CP from UP. These NFs are modularized to enable flexible Network Service (NS) development. The 5G CP follows the Service-Based Architecture (SBA) that provides a common interface for inter-communications among NFs. The 5G core architecture consists of eleven NFs with ten in CP while one in UP. Figure 1 shows 5G non-roaming reference architecture as

defined by 3GPP [1] .

· The ten 5GC CP network functions (NFs) in CP include Access and Mobility Management Function (AMF), Network Repository Function (NRF), Session Management Function (SMF), Unified Data Repository (UDR), Policy Control Function (PCF), Unified Data Management (UDM), Network Slice Selection Function (NSSF), and Authenticate Server Function (AUSF). In addition, 5GC also allows Application Function (AF) which enables service providers to create their own services cooperating with 5GC through the interaction with Network Exposure Function (NEF). The N1 and N2 interfaces support signaling exchange between UE (User Equipment) and AMF and between RAN and AMF, respectively.

· The only 5GC UP network function is User Plane Function (UPF) which is responsible for creating and maintaining PDU (Packet Data Unit) sessions to route and forward data packets to various external Data Networks (DNs). The N3 interface connects RAN and UPF. The N4 interface supports signaling exchange between UP and CP. The N6 interface is the connection to the external DN. The N9 interface supports signaling exchange between UPFs.

5G CUPS makes the deployment of modularized network functions of 5GC more flexible and agile. In our experiment, we aim to leverage the CUPS characteristic in 5G to create event slices [12] , defined as network slices created specifically to handle the sudden rise of traffic in special events such as New Year’s Eve party. As a result, we would structure our 5GC slicing experimental system with UP as customized event slices and CP as a common slice shared by those event slices.

There are not many 5GC open sources available currently. Most of the mobile network open sources such as nextEPC and OpenAirInterface only provide 4G EPC. Since free5GC [13] , an open-source project for 3GPP 5GC Release 15 and beyond, developed by NCTU in Taiwan, provides a complete 5GC architecture, we adopt it to construct our 5GC slicing experimental system.

ETSI NFV-MANO [14] is an architectural framework for enabling Network Services (NSs) (and thus network slices) under virtualized infrastructure. Each NS is composed of one or multiple VNFs. These VNFs are connected by Virtual Links (VLs) and their interactions are regulated by VNF Forwarding Graphs (VNFFGs). To deal with different QoS requirements of various vertical services, operators need to provide NSs of different characteristics.

As depicted in Figure 2, the ETSI NFV-MANO architecture consists of the following three major function components [14] :

· VNF Manager (VNFM): This is responsible for the lifecycle management of VNFs on the VNF platform, such as creation, modification and termination. Note that each VNF is a network function thus should be coupled with an Element Management (EM) system which is responsible for FCAPS¹ management

functionalities for VNF.

· Virtualized Infrastructure Manager (VIM): This is responsible for managing the NFVI resources including compute, storage and network. These resources are transformed from hardware resources to virtual resources via a virtualization layer. There are two types of VIMs: Virtual Machines (VM) enabled by OpenStack and containers enabled by Kubernetes.

· NFV Orchestrator (NFVO): This is responsible for the orchestration of NFV Infrastructure (NFVI) resources, the lifecycle management of NSs and the repository and management of deployment templates for NSs such as Network Service Descriptor (NSD) and Virtual Network Function Descriptor (VNFD).

MANO provides a framework to manage network slices. There are several MANO open sources available such as Tacker, OSM, OpenBaton, ONAP. We would leverage Tacker [15] as our NFVO/VNFM and OpenStack [16] as our VIM in our experiment.

· OpenStack is an open-source cloud operating system for virtualizing and managing resources including compute, network and storage. It provides multiple managing services such as compute (NOVA), network (Neutron), storage (Cinder), orchestration (Heat) and user interface (Horizon). We leverage Open vSwitch to create our experimental networks where switch virtualization techniques are used to manage the bandwidth limitation for slicing.

· Tacker is an OpenStack project that provides the functionalities of NFVO and VNFM to deploy and manage NSs and VNFs on either OpenStack or Kubernetes-based NFVI. Tacker is compliant to ETSI MANO Architectural Framework but requires some enhancements to smoothly orchestrate end-to-end NSs based on VNFs.

According to 3GPP, a network slice is an end-to-end network architecture consisting of multiple network slice subnets. Each network slice subnet represents a different segment in an end-to-end network such as access network, core network, transport network. Each network function can be deployed as VNF or Physical Network Function (PNF). NFV MANO, as the key enabling technology for network services, maps a 3GPP-defined network slice/network slice subnet to an NS. Figure 3 shows the mapping between a 3GPP network slice and an ETSI NFV NS [17] . The left side of Figure 3 shows 3GPP-defined communication service, network slice, network slice subnet, network function and their relations. On the other hand, the right side of Figure 3 shows NFV-defined network service, VNF, PNF and their relations.

The MANO-based 5GC slicing system can be divided into three parts: user plane, control plane and management plane. The ETSI NFV-MANO architecture is the management plane while the 5GC platform serves as the control and user planes. The management plane is responsible for the life cycle management of NSs, VNFs and other virtual resources. On the other hand, the 5GC supports all network functions required by the control plane and the user plane to provide 5G services.

In this research, we assume a 5G core with multiple slices pre-provisioned and instantiated. As 5G service traffic arrives at the core network, its control signaling will be directed to a common network slice that consists of all CP VNFs. On the other hand, its user data traffic will be routed to multiple customized UP slices preconfigured with different QoS characteristics, then forwarded to different data networks

Our slicing design is focused on customized slices. In three customized slices, only one UPF is deployed but each with a different bandwidth setting. These customized slices are designed to optimize resource allocation. Table 1 shows the bandwidth allocation and expected traffic of each customized slice. These three slices are to handle three different kinds of 5G eMBB traffic with low, medium and high bandwidth requirements.

In the compared one-size-fits-all single slice system, we set its user plane bandwidth as the summation of those in three customized slices of our proposed system. In the proposed system, we specify 3 vCPU, 16G RAM and 40 GB disk for each customized slice. In the compared system, we specify 9 vCPU, 48G RAM and 120 GB disk. All the UPFs in UP are equipped with the Debian 7 operating system. Consequently, the total virtualized resources of the compared and the proposed systems are exactly the same except that the proposed system would set the bandwidth limitation on each customized slice.

On the other hand, the common slice consists of 9 CP VNFs including AMF, SMF, PCF, NRF, UDM, AUSF, UDR, NSSF and mongoDB. Note that mongoDB is not the formal NF provided by 5G. Nevertheless, free5GC separates it as a storage NF for all other NFs. It is used to store the subscription and policy data in UDR, the network function profiles in NRF, etc. We configure all the VNFs in the common slice with 1 vCPU, 2 GB RAM, and 40 GB disk, equipped with the Ubuntu 18 operating system.

Before instantiating our proposed 5GC slicing system, both NSD and VNFD need to be designed [18] . The VNFD describes the behavior and deployment information of a VNF [19] . It may be constituted by the following deployment components:

· Virtual Deployment Unit (VDU): This represents the specification of a VNF Component (VNFC) that can be run on a VM or container. It includes deployment properties such as name, image, flavor and key. The image is the executable that can be instantiated on a VM or a container. The flavor includes disk, RAM, vCPU, etc. and the key specifies the key pair for authentication.

· Internal Virtual Link (VL): This is an internal VL providing connectivity between VDUs. In OpenStack, it specifies as a network.

· Connection Point (CPt): CPt is used to connect to the internal VL. It specifies the connectivity between Internal VL and VDU. We could assign the IP address for VDU or specify the security group for traffic management in CPt.

Figure 5 shows the relation between VDU, VL and CPt in Tacker. In the current implementation of Tacker, a valid VNFD must include VDU, VL and CPt. Each VDU needs to connect to a VL through a CPt. The communications between VDUs would go through the same VL. In our system, each 5G VNF consists of only one VDU and each VDU connects to a private network we create in advance to simulate the VL bound by CPt.

To enable network services, the NSD describes the behavior and deployment information of an NS [20] . It consists of:

· VNF Descriptor (VNFD): NSD would reference one or multiple VNFDs it needs.

· Virtual Link Descriptor (VLD): VLD would specify the connections between VNFs. We can call these external VLs to distinguish from the internal VLs between VDUs.

· VNF Forwarding Graph (VNFFGD): This defines the connection topology of VNFs in network slices. VNFFGD is usually used to create a service function chain.

In NSD, we need to import one or multiple required VNFDs which have been on-boarded and specify their topology. In our system, we deploy CP and UP as a network slice subnet respectively. The combination of CP and UP network slice subnets builds up the network slice of 5GC. In our system, we don’t have any service function chain. Therefore, there is no need to specify any VNFFG. In our implementation, we only instantiate the VNFs we need in NSD but no virtual links between VNFs (Tacker does not support VLD). Instead, all the 5GC VNFs would be deployed to a private network (Internal Virtual Link) we create in advance to simulate the inter-VNF communications.

Note that Tacker only supports VNFD and VNFFGD in NSD in its current implementation [21] . It doesn’t support PNFD (PNF Descriptor) and VLD in the NSD level for inter-VNF communications. Instead, the connection between VNFs is specified in VL and CP at the VDU level in VNFD.

We deploy the proposed 5GC network slicing system using MANO. Below we show the workflow of our system operations including the setup of the 5GC network slice environment, the onboarding of required NSD and VNFDs and the testing of the proposed system.

1) The MANO system is brought up to construct the 5GC network slice environment. Both VNFM and NFVO (Tacker) would be set up first then OpenStack needs to be registered as VIM to Tacker in order to manage the NFVI resources.

2) The VNFDs of all required network functions need be developed, then on-boarded to VNF Catalogue one-by-one through Tacker after verification.

3) The NSD of the network slice/network slice subnet based on the VNFs just on-boarded needs to be designed and on-boarded to NS Catalogue through Tacker after verification.

4) Once both NSDs and VNFDs have been on-boarded, we can proceed to instantiate the network slice/network slice subnet instance. The Tacker would create VNFs with proper requirements based on the NSD and its constituent VNFDs, then activate them.

5) After all the VNFs are up and running, we can configure the network bandwidth by CLI through VIM manually. This is to set the bandwidth requirements of the UP slices. The automation of this step is currently being developed.

6) We would register Traffic Generator simulating UEs to the 5GC to set up the testing sessions for data transmission to the specific user planes. Each UE is pre-assigned to a specific user plane.

7) After the PDU sessions are set up, UEs would start to send data simultaneously to simulate real traffic.

8) Measurement was done by testing the throughput and response time with UDP and ICMP packets under three different type of eMBB traffic: low, medium and high.

We followed the ETSI MANO NFV framework discussed in Section III to build our slicing environment. We employed two rack servers configured according to Table 2 to deploy Tacker and OpenStack. An NFV-MANO testbed is hence created to manage and orchestrate our virtualized slice environment.

The testing scenarios were sending two types of data: 1) ICMP; 2) UDP. UDP is used to create traffic load while ICMP is used to discover the response time. The whole experiment is done in the virtual environment. We create three traffic generators to simulate three sets of UEs generating three types of data traffic with different bandwidth requirements: High at 210 Mbps, medium at 100 Mbps and low at 50 Mbps.

Each traffic generator would register to the 5GC to set up the PDU session on UP. After the session’s setup, it would generate UDP packets toward a specific UPF. At the same time, ICMP data would be sent periodically to discover the response time. In the data network, we would set up a server to process the packet from the specific traffic generator. Figure 6 shows the data paths for the proposed system and the compared system. The UPF of the proposed system is sliced into three with different bandwidth requirements while the UPF of the compared system remains as one with the aggregated bandwidth of all UPFs in the proposed system.

We evaluate the performance by collecting throughput, response time and CPU utilization under the UDP traffic. Only UDP packets with upLink traffic were

sent. In our testing, we collected the measurements for the throughput by sending the UDP packets to the server in DN and the response time by sending the ICMP packets to the server in DN. The latter is calculated by the traffic generator as the round-trip time of the ICMP packet under three different types of UDP traffic.

Figure 7 shows the results of average throughput of the proposed system and the compared system under three type of traffic. The bars show the comparison of average throughputs under three types of data traffic: high at 210 Mbps, medium at 100 Mbps and low at 50Mbps. Overall, the average throughput of our proposed system is 1.27 times higher than that of the compared system because a UPF dedicated to handle each type of traffic instead of one UPF handling all three types of traffic.

Figure 8 shows the results of average response time of the proposed system

and the compared system. Here the average is calculated from all three types of data traffic. Since the response time of the compared system will increase dramatically when the traffic becomes very large, the proposed system gets a much lower average response time than the compared system. We suspect this was caused by the computational limit of the UPF design in free5GC.

In CPU Utilization, we only measure the user plane slices. The result of CPU Utilization test for the proposed system and the compared system are shown in Figure 9. We allocate the same resource and bandwidth limitation to both systems. Since the overhead of operating system and the heavier usage of computing resources with three UPFs instead of one UPF in the proposed system, the average CPU utilization of the proposed system is approximately three times higher than that of the compared system.