Docker images to echo HTTP headers and other details for testing purpose

When we want to test simple features in Kubernetes, we need simple web server images. We don't need a full-fledged docker image but something that responds with the IP address, HTTP headers, etc... That way we can quickly test the load balancing and other basic features.

Some create new docker images to do so. But we don't need to as this is a common problem and others have already created such images. The best way to search is with the keyword 'echo' in the docker hub.

Below are some echo images

• https://hub.docker.com/r/ealen/echo-server
• https://hub.docker.com/r/mendhak/http-https-echo/tags
• https://hub.docker.com/r/hashicorp/http-echo/

These are capable of responding differently according to the inputs given. It is really handy to simulate error messages, caching, etc...Check out for more details.

In case there is a better one, please comment.

Azure @ Enterprise - Scheduling task inside Windows container in Kubernetes

Problem

As part of my day job, we are doing a lift & shift of a giant monolith .NET 4.8 Windows-based application to Azure Kubernetes Service. Currently the application supports scheduling tasks that can be created and configured by the admins. Behind the scene, it creates scheduled tasks in Windows Task Scheduler pointing to the respective executable and passing right parameters.

How to do this in AKS or Kubernetes in general?

Solution

Since we need to limit the changes, we are planning to use the Windows task scheduler inside the container. Yes, it is going to be a Windows container and it may not be the most innovative solution in the world of containers where containers may be destroyed and recreated at any time.

Keeping the design things aside, this post is to explain how the Windows scheduled tasks can be created inside the container.

Nothing fancy, the same command that works inside a virtual machine to create a scheduled task will work inside Windows containers as well. Below goes the command.

[Video] Configuring container environment variables for ASP.Net WebAPI hosted inside Kubernetes on Docker Desktop

This is the second video in the Kubernetes series. Actually, when I started vlogging, I had decided that I will make short videos of max 5 minutes. But unfortunately, I was not able to stick to that. Here onwards I am planning to stick with my time limit. 

What is new?

We have to configure applications based on the environment it runs. It is not different for containerized applications running in Kubernetes. This video is showing how to configure the environment variables for the containers.

It uses an ASP.Net Core WebAPI application that returns the environment variable name when invoked with the name. It uses .Net 5 and basic YAML to deploy to Kubernetes that is running on Docker Desktop using WSL2 as a backend.

No more details. Please watch the video below.

References

https://kubernetes.io/docs/concepts/configuration/configmap/

[Video] Running simple ASP.Net Web API in Kubernetes on Docker Desktop

This post mainly contains the notes that I used to create a video. The video talks about hosting ASP.Net WebAPI into Kubernetes. The API can be accessible from the host machine using http://localhost:<port>/. The aim is to show simple hosting in Kubernetes that runs using Docker Desktop. 

It is advised to watch the video to get more clarity. 

Dev environment

Please refer to the previous post about setting up Docker Desktop and K8s environment. Along with that, it is required to have Visual Studio 2019 with docker container ASP.Net tools installed.

Source code

This post is not about creating a new visual studio project and enable the K8s support. Instead, download the sample from the below location

https://github.com/joymon/dotnet-demos/tree/master/web/webapi/simple-k8s-hosting/SimpleK8sHosting

Running using Docker Desktop

Just click on the run button in Visual Studio 2019 to run using Docker Desktop. The Dockerfile and other setups are already done in the sample.

Publish to Docker hub - the image registry

Below are the commands to tag and push to Docker Hub

Getting started with Kubernetes dev environment using Docker Desktop

This is 3rd or 4th time, I am learning Kubernetes (hereafter mostly refer as K8s short form) hands-on sessions. Every time I learn the kubectl command and its options, I forget as there were no chances to apply in the day job. Another mistake I did all those times was missing to post learning to this blog.

Hope this time I will get a chance to use it in the day job and not miss posting the Kubernetes learning on to this blog.

This post is very basic. The aim is to get started with the Kubernetes development environment using Docker Desktop. Below are the steps at a high level to get started. Detailed steps with videos are available on the internet.

Install Docker with Kubernetes support

The first step is to download the Docker Desktop and install it. It's straightforward as installing any other software in Windows. It is free and requires virtualization to be turned on. 

In order to run the Linux containers, it is better to enable the WSL2 backend. There are detailed instructions available to get it done including how to enable WSL2 on Windows.

Kubernetes support is only available when we use the Linux container mode. Note that Docker Desktop supports both Windows and Linux modes.

Points to note

• Better don't take the experimental build. Stay one version below the current stable version.
• This installation will modify our hosts file in Windows. Better not change anything.
• It is better to have an 8 core 16GB machine to work smoothly with container workloads.

Install Kubernetes dashboard

For beginners, it is easy to understand what is going on in the K8s cluster by looking at a UI dashboard application. K8s don't install a UI dashboard by default. That dashboard can be installed by using the below PowerShell commands.

---

SQL Server from container - Before jumping into the bandwagon

It is easy to get started on development and simple testing using SQL containers. It was discussed in the previous post. But before putting into production and start developing real applications we had to make sure the below things at least.

SQL Server running inside Linux container doesn't mean its free

People from the Linux world or in general people tend to think that everything in Linux is free. But that is not true in the case of SQL Server similar to other enterprise software running in Linux. In order to use the SQL Server container in production, we need a valid license even in the Linux containers.

Configuring and operating

Running and operating containers is an added skill to the operation teams. We need to make sure our client or in-house operational teams are capable of doing it.

Feature parity

There are many factors influencing feature differences. Microsoft always says its the same codebase but features don't reflect that. Let's see the factors that decide features

• Versions - Obviously there will be differences between SQL 2016, SQL 2017, and SQL 2019 features. The codebase is never the same though the latest may be built on top of the previous.
• Editions - First level feature difference inside the same version is based on the edition. This is regardless of wherever we run the SQL instance. This makes sense as we need to give more money to unlock features. This is there from the early days of SQL Server.
• Running platform - There are feature differences based on where the SQL instances run such as standalone inside Windows OS, Azure, and Linux containers.
• Azure PaaS platform - Inside Azure, there are Azure SQL and Azure SQL managed Instance models. They have many feature differences.

The containerized SQL server is more like the Azure SQL than compared with desktop SQL installation. Features such as FileStream, CLR support are not available in SQL container. The full list of features that are not supported in the SQL container can be found here.

Production readiness

As per Microsoft, SQL Server on Windows Container is not production-ready. But SQL Linux containers are production-ready. They have documentation on High availability when running SQL Linux containers and how to orchestrate using Kubernetes.  But not sure currently companies are investing to just move their SQL workload to a containerized production environment. 

Everyone agrees to start using it for development and testing. Eventually, when more workloads start using the SQL Server Big Data Cluster features, production will also start using SQL Server in containers. There are also documentation says SQL container as is don't support clustering hence the DR can be done only by log shipping. The Kubernetes based HA for SQL containers may not be instantaneous for an RDBMS database like Always On Availability Groups or FCI. It may take some time to detect the failure and start the next container instance.

As of now, my takeaway is not to use SQL containers for critical production workloads.

SQL Server from container - Getting started

Background

Historically not only installing SQL servers, but most of the software had difficulties. The container technology such as Docker magically solved this by packaging software as installed VM like images called docker images. It is easy as downloading the image and start using the software. 

There are a lot of ways to get it working for us if we want to get SQL instance up and running using containers.

Why SQL Server from a container?

Let us see the benefits of running from containers

Running with minimal efforts

No need to spend time installing and configuring. Below single command will get us started provided the Docker is preinstalled. 

This is really useful if we just need to test something out.

Deploying PlantUML Server container to Azure App Service

Background

PlantUML is a plain text language for writing diagrams. We just type the diagram specifications and diagram visual can be generated from the same. Easy to version control, easy to compare what changed, etc...
Visual Studio Code helps us to edit the diagram easily with preview and the environment is easy to setup. We can commit the diagrams to GitHub or any version control which can be rendered using PlantUML server. www.plantuml.com has a free service that we can use to render the diagrams from GitHub.com which even we an embed in web sites.

There is a separate tutorial on publishing rendered PlantUML diagrams using the www.plantuml.com provided rendering server.

Problem

The problem with using www.plantuml.com provided rendering is that it uses HTTP. No secure HTTP(s). An example URL is given below.

http://www.plantuml.com/plantuml/proxy?src=<URL to .puml>

There is an issue in the repo but not yet resolved.

Solution

One possible solution is to host the PlantUML server ourselves with our favorite hosting provider. If budget is a constraint, there are free hosting providers. Since its a Java application, we cannot host with GitHub pages or static file hosting providers. Another option is to host with cloud providers using their free tiers. Microsoft Azure has such an offering called App Service. They even allow containers to be hosted there.

This post is describing how we can host containerized PlantUML-Server in Azure App Service.

Steps

Below are steps to host our own PlantUML rendering server.

1. Create Azure Web Sites
1. Select Linux containers
2. Image source = "Docker Hub"
3. Repository Access = "Public"
4. Image and optional tag = "plantuml/plantuml-server:tomcat"
5. Create App Service
6. If there are issues in Refreshing the container image
1. Go to "Configuration Blade" look for "DOCKER_REGISTRY_SERVER_URL" setting
2. Change to https://hub.docker.com if it is https://index.docker.io
2. Prepare the URL
1. Normal URL - http://www.plantuml.com/plantuml/proxy?src=<URL to .puml>
1. eg:http://www.plantuml.com/plantuml/proxy?fmt=svg&cache=no&src=https://raw.githubusercontent.com/mind-maps/software/master/debugging/windows-network-debugging.puml
2. Azure App Service URL - https://<app service name>.azurewebsites.net/proxy?src=<URL to .puml>
1. eg:https://plantuml.azurewebsites.net/proxy?fmt=svg&cache=no&src=https://raw.githubusercontent.com/mind-maps/software/master/debugging/windows-network-debugging.puml

The only place where some confusion may occur is the container settings. Below is the screenshot to make things clear

Troubleshooting

Below are some issues faced during the initial setup. The steps above already taking care of the solution.

Docker registry server URL

When we create Azure App Service, by default it points to https://index.docker.io. As of writing this post, that shows it has plantuml/plantuml-server when we search. But throws HTTP 404 when we click on the link. But the https://hub.docker.com works fine. This issue is already in Docker Forums.

URL difference

Below is the URL if we use the proxy provided by plantuml.com

http://www.plantuml.com/plantuml/proxy?src=<URL to .puml>

When we host the PlantUML server the URL has to change the format is below for Azure App Service URL
https://<app service name>.azurewebsites.net/proxy?src=<URL to .puml>

The difference is that we don't need the /plantuml path just after the domain.

Linux Container not allowing to install packages - Selenium network debugging

It started with someone who needed to containerize Selenium. There is standard help available on how to do this. But less when something goes wrong. One such scenario is the network connectivity. In our enterprise environment, connectivity is a mystery and it played a villain role here too.

Selenium providing the debug version of the container image. We tried that to include the ping utility as ping is not part of the standard Ubuntu package.

Below is the Dockerfile we tried to install network utilities into the Selenium image

FROM selenium/standalone-chrome-debug:3.141.59-xenon

RUN apt-get update && apt-get install -y iputils-ping

RUN apt-get install dnsutils

EXPOSE 4444

EXPOSE 5900

Didn't worked. It was complaining about the permission to install. We never used any user name and password while doing anything in this exercise. So confused on how to give permissions. Also, we are mainly a Windows shop. Fewer experts in Linux.

Finally, google helped as usual. We switched to root by using the USER keyword in Dockerfile and get the utilities installed.

FROM selenium/standalone-chrome-debug:3.141.59-xenon

USER root

RUN apt-get update && apt-get install -y iputils-ping

RUN apt-get install dnsutils

EXPOSE 4444

EXPOSE 5900

The beauty of containers is that only one person has to struggle. The Selenium image with network utilities can be obtained from the below location.

https://hub.docker.com/r/joymon/selenium-standalone-chrome-debug-network

For a person from the Linux world working with containers for long, this may not be big news:)

References

https://github.com/WASdev/ci.docker/issues/194
https://github.com/SeleniumHQ/docker-selenium/issues/725

Software Containerization via Docker - First look and some thoughts

I was hunting behind the architecture of micro services and its advantages in distributed systems development and maintenance. The word 'Docker' stuck me when I heard that word multiple times in  one of the talk regarding micro services in Netflix. In that talk Adrian repeatedly telling that the Docker helped them a lot to deliver fast.They are not at all bothered by "works in my machine" scenarios. Then I started investigating what is Docker?

What is Docker

Docker is a software containerization technique where software with its dependencies can be run independently in an isolated environment. The container is run by container engine which is running inside Linux OS. If we are hosting an web application inside web server hosted in the container, we need to explicitly open ports for that container to establish communication. There are APIs to do container operations. The main advantage is friction less movement of executable software from one environment to another environment. If we make sure once our software is running inside the docker container, that container can run in any place where the docker engine is available

This easily leads to a question of

What is the difference between a virtual machine and docker container?

What I have understood from my analysis is virtual machine is an OS running inside another OS. It is heavy as the host operating system. If we want to utilize same hardware for multiple operating systems, we can go for that. But if we need mobility of software only better way is container concept.

Solomon, Founder and CEO of Docker Inc simply comparing docker ecosystem with logistics industry in his talk posted in www.docker.com. 

To me, the virtual machines looks like below.

But is this what we need when we need to transport software in its running condition? Absolutely no. That is where container comes into play as follows.

The vehicle which is carrying the package doesn't need to bother about what is inside container as long as it adhere to standard container specifications. In software, we really don't need to carry the operating system when we want to move our software. But we need to carry the dependencies of course.

Is there any standard for containers in software industry as in logistics

I don't think there is one right now. Docker can have it's own container file format and APIs to manage the containers where competitors can have their own. But a standardization is really required and soon that would evolve as everybody needs to move their software such as on premise to cloud and vice versa.

Some Docker facts

• It uses new API in Linux kernel
• Right now the Docker engine can only work on top of Linux
• If we want to host Docker engine in Windows or Mac, we need to use Linux VMs
• The containers are Linux specific. Though we can host .Net via Mono.
• It is used for easing the deployment and maintenance of distributed application. No meaning in running UI app inside docker container.
• There is docker app store where we can uploaded our containers or get containers from others. Same like other app store. The major difference is we don't need to install the app, instead just use the already installed and configured software by running docker container on docker engine.

Is Docker ready for production?

I think yes, if we are dealing with services hosted inside Linux server boxes. For Microsoft .Net community its not. Again it's my opinion after some research may vary from person to person and scenario to scenario.

Docker from .Net developer's view

As Docker is not yet ready to directly run on Windows platform, I don't think, its the time for .Net developers to enter into the Docker world. Of course they can try out docker commands and host .Net services using Mono etc... Another thing we need to notice is that there is no windows containers which can be run by the docker engine.

There is a tool called Boot2Docker available for Windows users to host a minimal Linux OS and run the Docker engine inside it. Once the engine ie runtime is available any docker container can be run inside it.

Again, the main bottleneck why we can't run Docker engine in Windows is the usage of Linux specific APIs to build the Docker engine.

The good news is Microsoft announced partnership with Docker to bring the technology into Windows ecosystem. As usual it seems the strategy is cloud first. But in near future we can get Windows Servers supporting docker containers out of the box. More than that, if a standard for containerization comes as part of partnership, that would be a mile stone in software industry.

Other software container technologies

Docker is way ahead in the container business. But there are competitors as well in the software container business. Rock-It from CoreOS, ElasticBox , DrawBridge from Microsoft to name few.

Links

http://en.wikipedia.org/wiki/Docker_%28software%29

Windows Command Line interface to manage Docker environment. https://visualstudiomagazine.com/articles/2014/11/19/new-windows-cli-docker.aspx
http://azure.microsoft.com/blog/2015/04/16/docker-client-for-windows-is-now-available/
https://docs.docker.com/installation/windows/

---

For those who are not able to follow what is Docker and containerization, try google or just wait for couple of months, you will certainly start hearing the word.