How containers actually works? Namespace, Cgroup, Kernel

Containers are the fundamental building block of modern software development but how much you know about the inner workings of this technology?

Here in this blogpost, I will try to explain the technologies and internals behind the containers.

Containers vs Virtual machines

Containers - "Container is Packaging of code and its all dependancy".

Virtual machine (VM) - "Virtual machines are essentially actual computers like laptops or desktops that have additional isolation thanks to a software layer".

In order to prevent the VM OS from interfering with the host OS, virtual machines use virtualization techniques to create virtual resources for vm such as CPU, memory, and storage with better separation.

Why did containers become well-known when VMs were already common?

To understand the need for containers, let us first look at VM challenges. As previously stated, VMs are essentially your physical computer with an additional layer of software on top of hardware known as virtualization (hypervisor). VMs are an isolated system with its own dedicated resources on top of the host OS. Because virtualization allows for the creation of own resources, it creates a more isolated environment with its own operating system, which eventually makes it more storage consuming, not as portable, and with less performance boost when compared to containers.

Containers, on the other hand, are portable, with increased performance and less use of computing resources. They work this way because, instead of creating full-fledged OS, they only use the host OS with some individually required components. Containers essentially use host operating system components to spin up resources such as namespaces, cgroups, and kernel. This has an advantage over VM in that instead of using all of an OS's resources, it only uses the components that are required, making it lightweight.

When you start a new container, there are several key Linux components that come into play. These include:

Namespaces Namespaces are a key component of containerization. They allow a container to create a separate view of the operating system's resources. For example, a container can have its own network namespace, which means it can have its own IP address and routing table. Namespaces can also be used for processes, users, file systems, and other resources. Containers use namespaces to create an isolated environment for running applications. When you start a container, a new set of namespaces is created for the container, which provides the container with its own view of the system resources. Each container has its own set of namespaces, including the process namespace, network namespace, file system namespace, and more.

Here are some examples of namespace commands related to containers:

• `unshare` The unshare command is used to create a new namespace and run a command within it. For example, you can use this command to create a new mount namespace and run a container inside it.

The sudo unshare --mount /bin/bash command allows a user to run a new bash shell with a new mount namespace that is separated from the original namespace. This means that the new shell has its own isolated view of the filesystem, separate from the rest of the system.

The output of this command on a standalone Ubuntu machine would be similar to the output on the local machine. It would start a new shell with a prompt like root@hostname:/# indicating that the user is logged in as the root user and is currently in the root directory of the file system.

However, since the mount namespace is isolated, any changes made to the file system within this shell will not be visible outside of the shell. For example, if a user creates a file or directory within this shell, that file or directory will not be visible in other shells or in the original namespace.

In comparison to the output on the local machine, there might be some differences depending on the system configuration, such as the hostname, system user and group IDs, and file system paths. However, the overall functionality and behavior of the command would be the same on both machines.

• `ip netns` The ip netns command is used to create and manage network namespaces. For example, you can use this command to create a new network namespace for a container.

This command creates a new network namespace in Linux with the name "mynamespace". A network namespace is an isolated network stack, with its own interfaces, routing table, firewall rules, and network devices.

The first command creates a new network namespace with the name "mynamespace". The second command lists all the network namespaces available, which now includes "mynamespace".

• `nsenter` The nsenter command is used to enter a namespace and run a command inside it. For example, you can use this command to enter a container's process namespace and run a command inside it.

Assuming we want to enter the namespace of the systemd-journald process, which has a PID of 634, we would run the following command

This will open a new bash shell within the namespace of the systemd-journald process. From this shell, we can run commands as if we were in the same namespace as that process.

These are just a few examples of namespace commands related to containers. There are many more commands and options you can use to manage namespaces and create isolated environments for your applications and services.

Control groups (cgroups) Control groups are used to limit the resources that a container can use. For example, you can use cgroups to limit the amount of CPU, memory, or disk I/O that a container can use. Cgroups are also used to manage the resources of the host system, ensuring that a container doesn't consume too much resources and cause other containers or the host system to slow down or crash.

Here are some examples of cgroup commands related to containers:

• `cgcreate` The cgcreate command is used to create a new cgroup. For example, you can use this command to create a new cgroup for a container.

This command creates a new cgroup named mycontainer and specifies that it should have two subsystems: cpu and memory. The cpu subsystem is used to limit the CPU usage of processes within the cgroup, while the memory subsystem is used to limit the amount of memory that can be used.

You can check that the cgcreate command has worked properly by verifying the creation of the control group directories for the specified subsystems. In this case, since we created a control group for the cpu and memory subsystems, we should see the following directories created under the /sys/fs/cgroup directory:

• /sys/fs/cgroup/cpu/mycontainer

• /sys/fs/cgroup/memory/mycontainer

To check that these directories exist, you can run the following command:

Finally, run the following command to check the cgroup's current configuration: The cgget command is used to display the control groups and their attributes of the specified subsystem. Here, we are using cpu and memory subsystems and specifying the control group /mycontainer.

This output shows that the cpu.shares attribute is set to 1024 and the memory.limit_in_bytes attribute is set to 536870912, which is 512 MB.

• `cgexec` The cgexec command is used to run a command inside a cgroup. For example, you can use this command to run a container inside a cgroup.

This command executes the command ulimit -a && free -m inside the control group. The output shows the usage of CPU and memory resources by the mycontainer control group.

These are just a few examples of cgroup commands related to containers. There are many more commands and options you can use to manage cgroups and control resource usage for your applications and services.

Kernel The kernel is the heart of the operating system, and it's responsible for managing the hardware resources of the system. Containers use the host system's kernel, but with some restrictions imposed by namespaces and cgroups. This allows containers to share the kernel and its resources, while also providing some level of isolation from the host system.

Lets explore more on namespace

In summary, containers are a fundamental building block of modern software development that provide a lightweight, portable, and efficient way to package code and its dependencies. Containers rely on Linux kernel features such as namespaces and control groups to provide isolation and resource management. Namespaces allow containers to have their own view of the system resources, such as network and file system, while control groups allow you to limit and prioritize the use of system resources, such as CPU, memory, and I/O, for a group of processes.

With the use of these kernel features, containers have become a popular technology for building and deploying applications. They provide a consistent environment that can be easily moved between different systems and platforms, making it easier for developers to build and deploy their applications.

In conclusion, understanding the inner workings of containers and the Linux kernel features that they rely on, can help developers and system administrators to better manage their applications and resources, and build more efficient and reliable systems. By leveraging the power of namespaces and control groups, containers can provide a scalable and efficient way to deploy modern software applications.

Thank you for your support! If you find my content informative and helpful, please consider subscribing to my page/channel and sharing it with others who may find it valuable. Your support is greatly appreciated and helps me reach a wider audience. Facebook:-> https://www.facebook.com/thenexthacker

Twitter:-> https://twitter.com/the_nexthacker

YouTube:-> https://www.youtube.com/@thenexthacker

LinkedIn:-> https://www.linkedin.com/company/thenexthacker/