[Video] Before debugging code, see this Mind map

We all debug our issues as well as other issues. Here is one mind map before we jump into debugging other's problems. At least that will help us organize and make sure our tasks are not impacted by helping others.

It uses PlantUML to write diagrams. There is a separate video on how to setup PlantUML in our environment.

Add your suggestions, questions in the video comment box.

Deploying PlantUML Server container to Azure App Service

Background

PlantUML is a plain text language for writing diagrams. We just type the diagram specifications and diagram visual can be generated from the same. Easy to version control, easy to compare what changed, etc...
Visual Studio Code helps us to edit the diagram easily with preview and the environment is easy to setup. We can commit the diagrams to GitHub or any version control which can be rendered using PlantUML server. www.plantuml.com has a free service that we can use to render the diagrams from GitHub.com which even we an embed in web sites.

There is a separate tutorial on publishing rendered PlantUML diagrams using the www.plantuml.com provided rendering server.

Problem

The problem with using www.plantuml.com provided rendering is that it uses HTTP. No secure HTTP(s). An example URL is given below.

http://www.plantuml.com/plantuml/proxy?src=<URL to .puml>

There is an issue in the repo but not yet resolved.

Solution

One possible solution is to host the PlantUML server ourselves with our favorite hosting provider. If budget is a constraint, there are free hosting providers. Since its a Java application, we cannot host with GitHub pages or static file hosting providers. Another option is to host with cloud providers using their free tiers. Microsoft Azure has such an offering called App Service. They even allow containers to be hosted there.

This post is describing how we can host containerized PlantUML-Server in Azure App Service.

Steps

Below are steps to host our own PlantUML rendering server.

1. Create Azure Web Sites
1. Select Linux containers
2. Image source = "Docker Hub"
3. Repository Access = "Public"
4. Image and optional tag = "plantuml/plantuml-server:tomcat"
5. Create App Service
6. If there are issues in Refreshing the container image
1. Go to "Configuration Blade" look for "DOCKER_REGISTRY_SERVER_URL" setting
2. Change to https://hub.docker.com if it is https://index.docker.io
2. Prepare the URL
1. Normal URL - http://www.plantuml.com/plantuml/proxy?src=<URL to .puml>
1. eg:http://www.plantuml.com/plantuml/proxy?fmt=svg&cache=no&src=https://raw.githubusercontent.com/mind-maps/software/master/debugging/windows-network-debugging.puml
2. Azure App Service URL - https://<app service name>.azurewebsites.net/proxy?src=<URL to .puml>
1. eg:https://plantuml.azurewebsites.net/proxy?fmt=svg&cache=no&src=https://raw.githubusercontent.com/mind-maps/software/master/debugging/windows-network-debugging.puml

The only place where some confusion may occur is the container settings. Below is the screenshot to make things clear

Troubleshooting

Below are some issues faced during the initial setup. The steps above already taking care of the solution.

Docker registry server URL

When we create Azure App Service, by default it points to https://index.docker.io. As of writing this post, that shows it has plantuml/plantuml-server when we search. But throws HTTP 404 when we click on the link. But the https://hub.docker.com works fine. This issue is already in Docker Forums.

URL difference

Below is the URL if we use the proxy provided by plantuml.com

http://www.plantuml.com/plantuml/proxy?src=<URL to .puml>

When we host the PlantUML server the URL has to change the format is below for Azure App Service URL
https://<app service name>.azurewebsites.net/proxy?src=<URL to .puml>

The difference is that we don't need the /plantuml path just after the domain.

Mind map - front end browser application development

Another mind map? Yes, that too about the web development. Not sure this will help anyone as there are a lot of mind maps if we google for frontend development mind map or mind map of front end browser app development. But most of those are not up to our convenience. For example with the advent of Webpack, we may safely avoid older module systems and task runners. 

In short, need is my own mind map which may not be useful to others as the existing mind maps are not satisfying me.

PlantUML based coded mind map on front end browser development

Below is the PlantUML based coded mind map on front end web browser development.

Please click on the image to get clickable svg format

Other mind maps

https://www.mindomo.com/mindmap/front-end-development-e999af6b49404020b63ca473111c4b02

Networking for developers - Mind map of tools and commands

This can be considered as the second part of a previous post on Networking for .Net Web Developers. The textual version of tools and commands is not that intuitive to use. Here we have a mindmap of networking concepts, tools, and commands for debugging basic networking related issues.

Click on the image to see the SVG version of the same. SVG supports clicking hyperlinks.

An interesting thing here is that this diagram is also created using PlantUML and kept in GitHub. URL below.

https://github.com/mind-maps/software/blob/master/debugging/windows-network-debugging.puml
Feel free to issue a PR, in case of suggestions and additions.

How to set up an environment for editing PlantUML can be found here.

Azure @ Enterprise - Microsoft Identity Platform OAuth Mind map

Security is always tough for developers. They know how to make the app work in their machine. But clueless about how it has to work in real production with the highest security possible. It becomes more difficult when they need to understand protocols and implementation details of standards such as OAuth, Microsoft Identity Platform, etc...

I am not an exception from the above-said developer group. Learned the Azure AD and its mechanism the hard way and finally decided to make a mind map out of the knowledge. 

It is attached below. Click on the image to open SVG version to click on the links.

Since it uses a textual drawing tool called Plant UML, the source code is available. It is present in GitHub. As we all know Azure is evolving very rapidly. Any changes are welcome in the form of a pull request.

Please find the post on how to work with Plant UML diagrams in Visual Studio Code.

More mind maps

https://medium.com/alexfilipin/azure-ad-overview-b7bb36b31239

Angular + ASP.Net WebAPI Application Performance Optimization - Index and Mind map

This is an index of things which needs to be checked, for performance optimization of a simple ASP.Net + Angular application. The simple application refers to application which read and write to SQL Server. If the application uses some other data store, still the points about other aspects are valid.

The details of each item would be either a link to posts in this blog or link to other resources. If someone is struggling to find where to start or struck on an optimization exercise,hope these questions will light a bulb in their brain. Sorry to say that it is very difficult to find an algorithm which applies every performance optimizations.

Browser side

• Do we need to support desktop and mobile browsers?
• Any possibility to avoid Internet Explorer and support only modern browsers:)
• Do we have separate mobile site with URL or same site is controlled via bootstrap or similar display technique
• Static assets
• Are those cached? If not can we cache?
• If those are cached what is the cache expiry?
• If those are cached how its done? cache-control or Expires header? Is the cache really working?
• Is the server properly returning http 304?
• Are they minified and bundled?
• Are the images optimized to have small foot print? 
• Are they served from CDN?
• Angular best practices applied?
• If it is AngularJS, what is the feasibility to upgrade to Angular

The network

DNS and routing

• Hows the DNS resolved to IP?
• Is the application available public or accessed only via enterprise intranet?
• What is the network connection band width between clients and server? also between web servers and database server(s)

Network devices

• CDN
• Any option for warming up cache?
• Security / Application Gateways which open packets and scan for attacks.
• What is the time needed for security inspections? Is it fast, if we bye pass gateway for testing?
• Network load-balancer
• Do we have load balancer? Are we reached the height of optimization and need scale out or scale up? In Azure, the best practice is to use LB as the machines gets patches and restarts periodically. Also follow the best practices to group machines so that all of those will not be restarted same time.
• What is the algorithm used for load balancing?
• Do we have sticky session need?
• Common
• Do we have logs available from these devices?
• Are the logs enough to confirm that the backing web server is taking time to respond?

Web Server machine

• Does the requirement of project is practical to the server? eg: One quad core, 8GB VM may never support 100,000 concurrent users even if its a simple company static web site
• Is the machine / hosting infrastructure in Azure or any cloud? If so do all the best practices followed and coded by considering cloud principles such as retry pattern?
• What is the status of CPU, Memory, Network etc.. during load test or busy hour?

IIS

• Logging
• Does the logging (time-taken field) says requests are taking high time? Please note that that time includes server processing + data transfer
• Are we logging any custom field which are big in size?
• Are the extra logging such as failed request tracking enabled?
• During a load test of busy session
• If we go to IIS worker process screen are we able to see many requests pending?
• What is the http queue length performance counter shows ?
• Are we able to identify one or sub set of requests taking high chunk of processing time other than rest of requests? This can be done by analyzing IIS logs.
• Static assets - Kind of repetition from previous section. 
• Are they cached properly?
• Are we using ASP.Net Bundling or compile time bundled? Are they served in minimized form
• Can we increase cache expiry?
• Protocols
• Any possibility to leverage http/2, if not done?
• Is it using web sockets?
• Authentication
• Is it using windows authentication? If so NTML or Kerberos? Kerberos is recommended for better performance
• AppPool Recycling 
• What is the AppPool recycling settings? 
• Is the recycling causing delay?
• Any ways to run mock requests so that the cache is populated before real user hit application?
• Throttling / Limiting
• What is the MaxConnection Setting and other IIS level throttling?
• Are there any performance counter showing connections getting queued and rejected?
• Timeouts
• Is there any manual test and load test attempted by reducing the timeouts to lowest possible settings? Some times high timeouts cause some requests to occupy entire system resources
• Payload
• Does the http compression enabled

ASP.Net Application

ASP.Net pipeline till our code

• What are the throttling settings at different level?
• Are there any custom HttpModules which may reduce performance? Any unwanted modules?

Our code

• Are there anything which can be cached instead of going every time to database? eg: any reference tables which change only by deployment.
• Are there any retries happening and succeeding after lot of attempts? Try reducing retry count to 3.
• Are there static variable usage and any potential for lock contention and memory leaks?
• Are the locks applied to lowest possible code section?
• JSON optimization
• Can the property names be shortened?
• Are the default values serialized into JSON?
• Are we doing any HTML rendering at server and returning as JSON property? Any possibility to make application as pure client rendered SPA?
• Do we have high amount of reflection?

.Net General

• Are there high amount of GC collections? This can be found from analyzing performance counters?
• What is the fragmentation level in LOH?
• Can we do NGEN?
• Any memory leaks in the application? eg: open connections, unwanted static things which may hold objects?

Database

• Does the applications logs showing that the queries are taking time?
• Does the SQL setup with possible best practices? eg: temp db, flags, connection limiting etc...
• Are there any subset of SPs always slow?

Mind map

Finally, there is a mind map created using PlantUML for easy reference. Since this is kept textual in GitHub and rendered on demand, this would be living document than the post.

Details on how to get PlantUML working is available in different post including a YouTube video tutorial.

Other similar links

This is not the first and last link on optimizing ASP.Net and Angular applications.

• https://www.c-sharpcorner.com/UploadFile/dev4634/performance-tuning-in-Asp-Net-web-applications/
• https://insights.daffodilsw.com/blog/10-proven-ways-to-optimize-your-asp.net-application-performance
• https://stackify.com/asp-net-performance-tuning/

Setting up PlantUML for coding C4 architecture diagrams - Video Tutorial

At last I published a video tutorial. As always its very difficult to get the first job done, if its a different type of job.

The tutorial is about setting Visual Studio Code with PlantUML to code C4 architecture diagrams  as explained in one of my previous post.

Since I don't have the right recording equipment and not a native English speaker, I used a text to speech service. It is AWS->Polly service. I didn't feel somehow the voices in Azure are good. The voice is US English Male voice named Matthew.

You tube link is given below and also embedded.
https://www.youtube.com/watch?v=Zt3Bj1HMJ8g

PlantUML - Code, Generate and Publish Mind-maps from GitPages

Introduction

This post mainly aims to provide an idea how we can use PlantUML to create mind maps and host them from GitPages without spending anything. It obviously need a computing device with browser and internet connection.

PlantUML

No need to have introduction again, as there were posts written earlier on the same. Just to summarize

• It is a textual representation of diagrams.
• Helps to version and compare changes applied.
• Can render in different visual formats from the same textual definitions.

Mind maps

It is diagramming model where information about one concept is represented in relation with other related topics. More on Wiki.

Coding Mind maps

The setup explained in previous post is enough for coding mind maps. PlantUML supports mind maps out of the box in latest versions as beta.

Rendering diagram

Though the ideal way to publish into web is HTML, there seems some issues in generating HTML files out of the box. So better get it into image formats and embed into HTML using <img> tag. 

If we are using our machines during development, it is easy to have control over the environment and install GraphViz and Java runtime needed for PlantUML. Now a days everybody uses continuous integration and delivery method. The CI/CD environments with full control mostly require fees and here we are trying to achieve without any cost. Let is see how to get the diagrams published in free?

One way is to use containers where entire ecosystem would be present and use it in CI/CD environment. Another way which we are going to see is via proxy service of PlantUML

Using the proxy service is very simple. The official www.plantuml.com site has a service hosted in below url

http://www.plantuml.com/plantuml/proxy?src=<accessible URL to the plantuml file>

This src query string pair accepts a source url where the PlantUML code file is present and returns converted image representation.

eg:The below URL gives the png file corresponds to the PlantUML file located at the path mentioned in src query string parameter
http://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/mind-maps/software/master/debugging/before-debugging-others-problem.puml

More details on controlling the behavior can be found in this post.

The moment we got the png back via an URL, we can easily publish using html files. No need to have CI/CD systems and dealing with containers or build environments.

Publishing

GitPages provides free sub domain based web site hosting free and we can host mind maps directly from the GitHub repository. Using the markdown, we can easily convert the above mentioned url into an link <a> or <img> tag.

eg:A link in markdown file (.md) can be denoted as below

[Debugging](http://www.plantuml.com/plantuml/proxy?cache=no&src=https://raw.githubusercontent.com/mind-maps/software/master/debugging/before-debugging-others-problem.puml)

Sample repo

Source repo which hosts the puml files & hosting .md files - https://github.com/mind-maps/software
Sub domain & URL based web site which GitPages gives for repo- https://mind-maps.github.io/software/

The .md files gets automatically converted into HTML and served. If there is index.md it would be taken as default when we didn't mention anything in the url. GitPages internally uses Jekyll and customization can be done. One such customization in the above repo is in below URL. Since Jekyll and customization is not in the scope of this post, please google for more details.

https://github.com/mind-maps/software/blob/master/_layouts/default.html

Below are not essential but just to give complete picture
Sub domain based organization URL - https://mind-maps.github.io/
Repo of organization web site - https://github.com/mind-maps/mind-maps.github.io

Enjoy documenting...

Node-PlantUML package to automate generation of mind map diagrams

Recently I was doing some experiments with PlantUML. PlantUML is awesome. Write code and get diagrams. It is especially great tool for engineers like me who are not that good at art. One of the main trend now a days is to automate anything and obviously we can automate diagram generation from puml file. The technology to do is obviously NodeJS.

Package to generate images from .puml files

As normal in Node world, NodeJS has a package to convert puml files into images and much more. Below is the link to the NPN package.

https://www.npmjs.com/package/node-plantuml

From here, this post assumes the reader has knowledge about NodeJS and its NPM package ecosystem.

A simple problem

If there is no problem this post is not needed. The node-plantuml package generated the diagrams for most for the puml files. But not for the mind map pumls. ie which are starting with @startmindmap.

@startmindmap

* root

** 1

**_ 3

@endmindmap

Root cause

The node-plantuml uses plantuml.jar file to bundle with it is not the latest. That jar file don't have the capability to understand @startmindmap. So it fails.

Solution

node-platuml package to include the latest plantuml.jar file. As a responsible developer, who make use of other's open source work, I should give them a PR which I will be doing soon.

Workarounds

Set the path of plantuml.jar

As any other developer, the developers of this package also left option for configuring the path to jar file. They selected the environment variable mechanism to set the path. The name of environment variable is PLANTUML_HOME. If the environment variable is not there, it will take jar file from vendor folder. Below line is from the node-plant package source code.

https://github.com/markushedvall/node-plantuml/blob/master/lib/plantuml-executor.js#L9

The code to set the environment variable to a different .jar file is as follows. If we obtain the latest plantuml.jar file, we can place it somewhere and point to the same.

process.env.PLANTUML_HOME = "node_modules/node-plantuml/vendor/plantuml_new.jar";

If there are any troubles setting the path, try the below approach.

Replace old file with new

Get the latest plantuml.jar file from VS code extension called PlantUML. The location of the platuml.jar file is below.

%userprofile%\.vscode\extensions\jebbs.plantuml-2.10.9

Put the file into the node-plantuml modules folder. Location below.

..\node_modules\node-plantuml\vendor

Setting up PlantUML for coding C4 architecture diagrams

Background

Earlier there was a post in this blog related to documenting architecture using C4 Architecture model. The architecture of www.joymononline.in is documented in the C4 way. At least, the context diagram. The problem what was tried to solve in that post was how to make code inline with architecture.

It is easy to document architecture for a static portfolio site or a simple CRUD operations site with a login feature. But difficult in Enterprise systems especially, if the rate of change is so rapidly. Nowadays the software is changing its happening at high speed with the advent of DevOps culture and practices such as continuous integration, delivery, monitoring, etc...

Problem

When we follow any documentation model as a technical person, we may face below problems

• Versioning with application code.
• What if a feature branch changes the architecture or component interaction or alters dependencies
• Comparing what changed from the last version?
• The time-taken to beautify the diagrams. Skills depend on person to person and difficult to get it uniform when the organization grows.

Potential solution

One way we could apply is coding the architecture. Meaning, store the architecture diagrams in textual representation and store along with code. Generate diagrams out of it. It is a simple idea similar to using markdown for documents. It's human and machine-readable.

Let's see one way to document the C4 architecture model using textual language. First introducing of some terms and later see how they are smartly glued to get coded C4 architecture diagrams.

PlantUML

This is a text-based UML modeling. This has specs and free to use. It has also Java-based rendering mechanism which uses another free software called GraphViz to visualize the diagrams.

GraphViz

This is a visualization tool for creating diagrams. Its open-source. This is needed by PlantUML to render charts.

VS Code & PlantUML extension

This is the developer's favorite editor. As usual, there is an extension to easily write and preview the diagrams. The name of the extension is 'plantuml'. We can either install via extension editor or by command. Thanks to the author Jebbs for the extension and making open source.

How to setup VS Code for PlantUML is already mentioned in the below link. Since we are living in the 21st century, the Japanese language in the post can be converted automatically by the browser to English:)

https://ameblo.jp/sijukara-tama/entry-12445811317.html

C4-PlantUML GitHub repo

The magic is here. Below repo has the C4 customization files for PlantUML to understand the C4 language.

https://github.com/RicardoNiepel/C4-PlantUML

Big thanks to Ricardo for his work and especially for open-sourcing the same. The repo has enough details about the capabilities and sample to get started.

Another interesting thing he did is the availability of Azure customization. These files help us to create diagrams using Azure symbols than boring boxes.

https://github.com/RicardoNiepel/Azure-PlantUML

Troubleshooting

Though some of us get the environment working in the first attempt itself, I am trying to do a video (my first tech video) on how to set up the environment for doing C4 architecture of www.joymononline.in via PlantUML notations and the tools mentioned above

If we go step by step there will not be any issues. Below are the steps

1. First, install Java. That will make sure the PATH variable is set properly.
2. Then install GraphViz. Even there is no issue installing in a different order as those are not dependent. 
3. Now install the VSCode and 'plantuml' plugin. the plugin contains the plantuml.jar file. 
4. Finally, download the customization from C4-PlantUML repo and open in VS Code. 
5. Check the preview by Alt+D shortcut.

Once we have the environment setup its easy as typing the diagram and previewing it in the same window. Commit the file once ready. Better integrate with the build pipeline.

Conclusion

The mix of specs & tools is little complex but easy to set up and use. Though there is no support for the tools we as developers can fix the issues and add features in those tools and give back to the community