I remember my first times writing some code in C years ago. I was struggling to discover why my code got segmentation fault. To detect the point of failure, I was printing every single variable to console at some specific checkpoints which hopefully would tell me why. Back then, I wasn’t aware that my silly printf s were primitive examples of manual instrumentation.

What is instrumentation at all? It can be basically defined as rewriting the code in a way where you can gather meaningful data about it. The executed code in this way will not only do its job but also generate data for monitoring, just like my printf functions in my 40 lines of code

Why call it manual, then? Is there an automated version, too? Yes, there is. Instead of writing monitoring code inside the application logic, it is possible to embed monitor logic using parametric variables in a separate place without touching the application code. This is called automated instrumentation. This is called automated because the code to gather monitor data is generated automatically by an agent parsing the code and embedding necessary monitor codes.

Automated instrumentation has a serious advantage over manual: Manual instrumentation tends to cause errors in the application. There is a risk of running erroneous invocation while trying to get some insights about the code. On the other hand, automated instrumentation works more like a monitoring code generation machine which has been tested before so it is more reliable than manual.

Instrumentation in AWS Lambda

AWS Lambda functions run in arbitrary containers which are launched by AWS Lambda. For this reason, you cannot put an agent to a specific container to embed monitoring code and collect monitor data because you don’t know which container will host it next time. Automated instrumentation using an agent is not possible for this reason. Manual instrumentation is doable by embedding monitoring contexts into application logic. However, this is not the best option considering the reliability issue mentioned above. On top of that, the added part might extend the invocation time unintentionally if this part includes ineffective operations. This directly means cost for AWS Lambda users. Therefore, Using AWS Lambda in production may cause errors and/or performance degradation.

We need a safer and more reliable way of instrumenting our AWS function automatically. Good news is that there are two possibilities:

• Using the abstract runtime environments like JVM to relocate agents. It is not possible to guess which container will run my code but we can understand the virtual machine running the code.
• Embedding monitoring code to the application just before deploying to AWS Lambda by intervening in building of the code.

According to the runtime environment used, one of those is preferrable. We decided to go with the first option with Java runtime because we have JVM in our hands on which we can relocate our monitoring agent. For Node.js on the other hand, we will go with the second option since we don’t have an virtual environment like JVM.

How to make automated instrumentation with Thundra

We advocate strongly for automated instrumentation for application performance monitoring. We put in every effort we can to make automated instrumentation easier for this reason. In Thundra, it is possible to automatically instrument your AWS Lambda functions environment variables and/or annotations (this is applicable only for Java). In the two functions below, you can see the same instrumentation made by environment variables and annotation respectively.

At each option, we didn’t add any line of code to UserService class but we managed to instrument it to see the trace charts and logs of each line of the code as follow. This is what we call “no code change”.

There are many other configurable levels of monitoring such as monitoring function arguments, return values, errors, and more. All of those can be achieved with no code change. You even have the flexibility of starting to audit your function if it takes longer than a threshold.

It is also possible to put your own monitors manually to your Lambda function with Thundra. You can embed your own context and variables to watch. In the following example; you can see the variable called “hit” is defined in custom context.

For all of the possible configuration levels with automated and manual instrumentation, you can check our documentation for Java. Our Node.js automated integration will be available very soon! Stay tuned for more news!