Tuesday, November 23, 2021

DevOps - Importance of deleting container images by CI pipeline after pushing to registry

When we use self-hosted Azure pipeline agents, we may encounter the below issue during the build process. This is not a hard issue to troubleshoot. The reason is there in the error message.

Error processing tar file(exit status 1): open /root/.local/share/NuGet/v3-cache/670c1461c29885f9aa22c281d8b7da90845b38e4$ps:_api.nuget.org_v3_index.json/nupkg_system.reflection.metadata.1.4.2.dat: no space left on device

##[error]Error processing tar file(exit status 1): open /root/.local/share/NuGet/v3-cache/670c1461c29885f9aa22c281d8b7da90845b38e4$ps:_api.nuget.org_v3_index.json/nupkg_system.reflection.metadata.1.4.2.dat: no space left on device

##[error]The process '/usr/bin/docker' failed with exit code 1

Solution

Clean up the artifacts from the build server once it is uploaded to the permanent artifact's location. Also, clean up any residual that is produced as part of the build process such as unit test results. In our case, we faced this issue due to the leftover container images in a Linux build server. We have to delete the local images after pushing to Azure Container Registry.

Workaround

By default, Docker files (containers, images, etc) are located in /var/lib/docker which is part of the / filesystem. We can check how much space is left by using the df -h Linux command.

There are at least 2 workarounds

Option 1 - increase storage

Increase the size of the current / file system. This is easy if the build server is in the cloud environment. There will be some downtime of course.

Option 2 - change docker storage location

Mount new storage to the build server and point the docker to use that as the storage location. Below are the steps.

  • The default location of docker images is specified in the /etc/docker/daemon.json file. The property name is "data-root"
  • That can be changed to the newly mounted location.
  • Restart the Docker daemon.
    • sudo systemctl daemon-reload 
    • sudo systemctl restart docker
A totally different option is to get rid of self-hosted agents and use the Azure agents. This depends on the project requirements and enterprise policies.

References

Tuesday, November 16, 2021

Azure @ Enterprise - AAD App registration - Should I need it if "Admin consent required" is "No"

Security is hard. Especially every vendor says they follow the standard of oAuth and implement differently. Let us look at the Azure App registration's API permissions blade. 


<image permissions blade>

We can see there are 3 columns that may confuse us. One is the Type and 2 at last that denote whether the permission requires admin consent and the status column showing whether admin consent is granted or not.

Further, it reveals something very interesting. The 'User.Read' permission doesn't require admin consent but it's given. Why in the world someone needs to do like this?

Let us try to unmask the mystery or just read the docs.

Delegated permission v/s application permission types

It's very simple. The delegated permission is only for users and the application permission is for applications. To make it more simple, 'Delegated' means the users can delegate an application to do something on behalf of the user. Here the application is not acting as an application instead of as a user.

Application permissions is allowing applications to perform something as if it is done by the application. eg: scheduled tasks, queued tasks in general daemon apps

Column 'Admin consent required'

This means the developer or the publisher of the permission put a requirement that this permission is tenant-wide and important. So it requires Admin consent.  Let us consider 2 scenarios where admin consent is not required.

What if admin consent is not required and it's not given and used by the user app

When an application demands permission that doesn't require admin consent and is used by an interactive user application, there will be a consent form that pops up that asks the user to give consent

In short

The user has to give his consent even if the admin consent is not required.

What if admin consent is not required and it's not given and used by the daemon app

Here the app will fail because that application is not running under user context and there is no one to give consent.
Admin consent is always required for the daemon apps to work with application permissions.

This completes the answer to the title question. Yes we need admin consent for the daemon applications even if "Admin consent required" is "No" 

What if admin consent is not required but given and used by the user app

If the admin has given the consent, there is no need for the user to give consent again as its tenant wide permission. The consent popup will not be shown.

Special cases

ROPC Flow

This refers to Resource Owner Password Credentials flow where the application can act as a user by obtaining the user's credential (password). It can obtain the credential from its own configuration file when running as a daemon application or from the user when it's running as a user-facing UI application. 

When it is running as a daemon we normally create a service account and keep its credentials in the application configuration. In both cases, admin consent is required.
Whatever the case the ROPC is not recommended by MSFT

Restrict users from consenting

Even if a publisher application doesn't require admin consent the AAD tenant admin can override that to enforce admin consent for all permissions. This leaves users with no permission to give consent.

ReadTheDocs

https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-permissions-and-consent#using-the-admin-consent-endpoint

https://docs.microsoft.com/en-us/azure/active-directory/develop/application-consent-experience#app-requires-a-permission-within-the-users-scope-of-authority

https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/configure-admin-consent-workflow

https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/grant-admin-consent

Tuesday, October 26, 2021

Can we read console inputs in .Net Core BackgroundService?

Alert: This is a basic question-answer post for my tailored googling purpose. you may easily skip this post.

Background

.Net Core API model simplified many things such as dependency injection, logging, etc...One major thing is the background service to perform background operations. There are many use cases for BackgroundService. More of how to use BackgroundService and its interface IHostedService and how to host can be found in the official documentation.

Question

When we use BackgoundService, one simple question may arise in our minds. Can we get user input from the console by the code running inside the BackgroundService?

Answer

The answer is yes we can get the inputs from the console. There is a sample available in my GitHub repo that acts as a template for .Net console apps.

Why this post?

This post is mainly for my tailored googling purpose. I got this question 2-3 times. Next time I can search it easily by prefixing joymon😀

Interesting links regarding BackgroundService hosting

https://docs.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/background-tasks-with-ihostedservice

https://medium.com/@daniel.sagita/backgroundservice-for-a-long-running-work-3debe8f8d25b