Tuesday, August 27, 2019

Networking for .Net Web Developers

It is a common scenario web developers have to deal with networking issues. Sometimes they surface as functional issue, security or performance issue. If they have the background of computer science study stream, they are expected to know how the computer network works. But most of the developers in the software industry don't have computer science major. Also, even if they are, they might not have practical knowledge of networking. This is talking about mass development consulting scenarios not the product development companies where there are a lot of filtering during the interview. Quora has a question on whether the web developer should know the networking concepts.

However, when a developer grows into the technical lead then architect positions they should know how the network works. So it is better to start understanding the networking at the earliest.

Below are some concepts and tools with a simple description. The tools are either inbuilt in OS or completely free.

Tools & commands

Browser F12

This is the first place to look. Just press F12 button to open the developer tools in the browser. All modern browsers support the developer tools. It helps to identify how many resources downloaded to render a web page, the time-taken, and size for each request finally errors in JavaScript execution. It also helps to understand the time browser spent for rendering after the resources are downloaded.

Fiddler

This is acting as a proxy between browser and web server. The scenarios which are difficult to identify via Browser F12 tools can be found using Fiddler.
eg: If we want to simulate the presence of header or want to understand http compression is really present we can use fiddler

Wireshark

This is a little more advanced tool but comes with more power to analyze at low-level.

TCPDUMP

This a tool for collecting network dumps from production machines where Wireshark is not available. 

NETSTAT

This shows the current statistics of network connections. Inbuilt with Windows Os.

Resource monitor

Resource monitor tool in Windows OS can also show the connection stats. Open Resource Monitor and goto the 'Network' tab to see the network statistics

Logs

Let's look at the logs what we need to be aware of when troubleshooting network issues. The advantage here is that we can look at what happened. The monitoring tools above mostly shows what is the current situation.

Web server / IIS logs

It is better to start with the server logs which says what were the URLs users requested from the system and how the system responded.

Http.sys logs
FREB - This is IIS feature where we can see how the IIS pipeline executed the individual request. Good for troubleshooting authentication issues.

EventViewer

This is an excellent place for seeing what happened over time at the system level. There are inactive log sources that we can enable if the issue is not reproducing on-demand and expecting it will reproduce later in time.

Protocols & standards

Below is a list of protocols and standards to be aware. This is a growing list and applicable based on each application's nature. They operate at different layers of network communication. 

Must have

  • UDP
  • TCP/IP
  • HTTP
  • HTTP/2
  • SSL/TLS 1.2
  • SQL - TDS

Nice to have (as applicable to our application)


  • Authentication - OAuth, OIDC, Kerberos, NTLM
  • Files - FTP, SMB
  • Mail - SMTP

HTTP standards

These are some common standards HTTP has though they are not really low-level networking.
  • Caching headers
  • ETag
  • HTTP Compression 
  • Content-Types
  • Mime-Types

Courses

https://www.udacity.com/course/networking-for-web-developers--ud256

Conclusion

This is not a comprehensive list. It covers only the tools in Windows OS. The tool names may be different if the development platform is different.

Tuesday, August 13, 2019

System.Diagnostics logging in .Net - Trace sources

As developers, we often need to work with network issues. The first step is to finding out there is a network issue such as delay or package loss. This can be found by looking at Wireshark. But Wireshark needs some expertise. Also, it needs installation and mostly a restart after installation. When working in production environments it would be difficult to restart. But the other side is we can analyze without any changes to the application as setting diagnostics source needs application config changes.
But if we just consider the expertise needed for Wireshark, a better way is to collect some svc logs using Diagnostics. If required go for Wireshark as the second step of the analysis.

Diagnostics sources related to Network

We are not going to create another list. Below are the links to already existing tutorials

General networking

https://docs.microsoft.com/en-us/dotnet/framework/network-programming/how-to-configure-network-tracing
https://blogs.msdn.microsoft.com/waws/2017/03/14/how-to-get-a-system-net-trace-of-your-c-web-job/

WCF

Legacy ASMX

Other diagnostic sources

Since diagnostics source tracing is .Net feature, there are other diagnostics sources as well useful in different scenarios. Below are some collections.

https://stackoverflow.com/questions/2464071/what-is-the-list-of-trace-sources-supplied-by-the-net-framework

Unfortunately, it seems there is no official link of the full list of trace sources.