Introduction
Writing an article for an annual review has become a tradition for me. After writing for 7 consecutive years (link), I decided to continue this year, to share with you my journey in the tech industry, as a software engineer at Datadog, and as an explorer for other side projects during my free time. Hopefully, they will let you learn something new or inspire you to create your story. Now, let’s get started!
Datadog
I joined Datadog at the end of 2019 as a software engineer working for the Event Platform. This year, I was part of the automation team, which improves the platform’s reliability, reduces toil, and facilitates engineers’ daily operations. My contributions are around automation development and can be summarized in 5 parts:
- Developing the new generation of CI/CD tool “C” at Datadog. Tool C is a state machine that handles the full system development life cycle (SDLC) of all services at Datadog. It includes triggering the CI, building and releasing the artifact, handling custom transformation, verifying the compliance of the artifacts, and deploying them to the target environment. This project is a collaboration between multiple teams and it lasts for multiple quarters. It consists of the backend (core) logic written in Go (Temporal), the CLI to operate the executions, integrating to existing tools (GitHub, Slack, Service Catalog, …), and the frontend. This tool is used for more than 100 services and tens of teams at Datadog. This is the project that I have spent most of my time on this year.
- Migration of a legacy tag. This epic is for migrating a largely used legacy tag to its alternative in our system. This is widely used in many pieces: in the infrastructure (databases, Kubernetes clusters, cloud resources, …), in the data (metrics, logs), and in the metadata. Migrating means measuring the usage of the old tag and the new tag, providing alternatives to the query system, and leveraging automation (automatic deployment, automatic node replacement, …) for automatic rollout, reporting, troubleshooting, and collaboration within the team and other teams.
- Development of a gRPC client for a microservice. It allows communication between the workflows and our administration tool via gRPC. Having this SDK provides a clear interface for the data structure and provides a framework for other engineers to develop additional features with minimum effort. This gRPC client is designed using the domain-driven design (DDD). Later, this tool was used by other clients of the administration service.
- Integration of an internal CI/CD SDK for the core storage systems. The CI/CD team developed a new feature “T” to fit the missing feature required by core storage teams. This feature T quietly became a critical part of our CI/CD system. My mission was to collaborate with the CI/CD team to test it, roll out the feature T into our system, and provide feedback to ensure that the development matches peoples’ expectations.
- Other work. It includes reviewing automation systems, promoting CI/CD tools, interviewing, on-call, presenting, onboarding new team members, etc.
Overall, the work here at Datadog is really interesting and I learned a lot while developing the features above.
Learning
This year, I spent quite some time learning different things. Here are some topics that I want to share with you:
- Kubernetes Administrator. I wanted to learn more about Kubernetes. That’s why I followed a course on Udemy and passed the Certified Kubernetes Administrator (CKA) exam. It allows me to better understand the important aspects of Kubernetes and be familiar with the networking, configuration, worker nodes, master nodes, storage, security, and many more. Overall, it has a very positive impact on my daily work. I highly recommend Mumshad Mannambeth’s course Certified Kubernetes Administrator (CKA) with Practice Tests.
- Communication. As I gained more work experience, I found that communication was really important. Work is an activity that requires group cooperation. Whether it is your colleagues, your boss, or your users, you need to communicate with them and convey useful information. Many times, technology is not the real bottleneck. The real bottleneck is that requirements are not understood, priorities are not agreed upon, everyone’s goals are not aligned, etc. These can all be optimized through communication. Technical work is often complex and delicate, and only a few people know the implementation, which makes communication even more important. Learning how to communicate, how to listen to each other, how to express oneself, and how to say the right words at the right time has become a must-have skill. Stanford University’s podcast “Think Fast, Talk Smart” shared a lot of useful knowledge and benefited me a lot. You can listen to this podcast for free on YouTube.
- Management. Although I don’t have a management position at Datadog, I feel like learning the management basics is really helpful for understanding how my managers think about things and being aligned with them. I found this YouTube channel Bernd Geropp useful as he shared a lot of management basics with his subscribers.
To be more efficient in time management, I usually watch or listen to YouTube videos when commuting to the office or going gym (cycling on an indoor bike), in that way, I can do two things at the same time and only carefully listen to the content again if I find something really important.
Finance
The finance toolkit is a small library that helps you to understand your personal financial situation by extracting, transforming, and aggregating transactions from different companies into a single place. The companies supported are BNP Paribas, Boursorama, Revolut, and some others. It generates CSV files that can be used for data visualization. It was created in 2019 and written in Python by Jingwen Zheng and me, with some help from Mickaël Schoentgen. This year, we didn’t have any evolution on this project, but we are still actively using it. This tool helped us increase our assets by 12 times over the last 5 years.
Conclusion
Thank you for reading this article. In this article, I shared some projects that I did at Datadog as a software engineer, my learning experience, and the finance toolkit that I open-sourced last year. Interested to know more? You can subscribe to the feed of my blog, and follow me on Twitter or GitHub. Hope you enjoy this article, see you next time!