How to become a Machine Learning Engineer ?
Recently I posted a job post for a machine learning engineer position in my team @ Careem. I got a lot of questions from people who are curious to know more about the position itself, how to prepare for it and what kind of problems does it face ?
Data Science vs Machine Learning Engineering
Let's start with defining the position itself. What is machine learning engineering ? Is it a new thing ? Does it differ from the so-called sexist job in the 21th century , the data scientist ?
When people invented the term (Data Scientist), they were looking for superheros who excel at the following :
via Big Data [sorry] & Data Science: What Does a Data Scientist Do ?
1- Acquiring the data, cleaning it, identifying problems and asking the right questions as well as setting the correct measures for different KPIs.
2- Doing necessary statistical analysis to come up with different hypothesis about the solution
3- Using statistical methods to validate the different suggested solutions.
4- At some point, the solution could be more than a mathematical model (equation), it could include some complexity requiring the usage of machine learning. Thus, developing a suitable ML model through different iterations become necessary.
5- Once this model is ready, you need to design an engineering system that will serve this model in addition to other aspects that requires careful engineering to ensure continuous effectiveness of this model.
6- Do the necessary continuous analysis and investigations for the model results as well as the must-have monitoring.
Truly they exist on our planet but they are very rare ;)
Since human-beings are limited in their ability to acquire knowledge and to keep up with the non-stop changes in each different sub-field. Specialization became a must. We started to see different job profiles working on different areas of the pre-mentioned workflow.
For example, some data engineers/scientists who are more excited about infrastructure, specialized in data engineering. This gives them more bandwidth to absorb the internals of different distributed processing systems (ex: Apache Spark, Apache Hadoop, ..etc) and thus be capable of extending these systems to handle very custom use-cases that may not be easy to handle with the standard APIs
We can list 6 profiles that together compose a capable data team able to push its organization to be data driven and then AI driven at a later stage.
Data Analyst & Data Viz
Data Scientist
Data Engineer
Machine Learning Engineer
Data Infrastructure Engineer
Data Ops
Still, it is hard to generalize this to every single organization as the needs and size of each are the deciding factors while considering the boundaries between each profile under the data team umbrella.
So What is a machine learning engineer ?
I will quote the excellent article from Oreilly which describe them as the following :
- They have stronger software engineering skills than typical data scientists.
- Machine learning engineers are able to work with engineers who maintain production systems.
- They understand software development methodology, agile practices, and the full range of tools that modern software developers use: everything from IDEs like Eclipse and IntelliJ to the components of a continuous deployment pipeline.
- They are involved in software architecture and design
- Because their focus is on making data products work in production, they think holistically and factor in components like the logging or A/B testing infrastructure.
- They are up to speed on issues that are specific to monitoring data products in production. There are many resources on application monitoring, but machine learning has requirements that go even further. Data pipelines and models can go stale and need to be retrained, or they might be attacked by adversaries in ways that don't make sense for traditional web applications.
How to get the experience ?
Photo by Alexandru Acea / Unsplash
- Be a solid software engineer
- Get ML experience
- For the theoritical part, you can take any of the existing MOOCs on Coursera, Edx or Udacity. One option is Udacity Machine Learning Engineer Nanodegree.
- Get practical experience through doing real projects on real data. Kaggle is the best source for problems if you don't have some already :)
- Read, Listen and Watch. There are dozens of awesome articles, podcasts and video sessions that are available on the internet. You can also join many subreddit that gives you updates about different ML topics.
Recommended resources
Blogs
Towards Data Science
Machine Learning Mastery
[Learn With Passion] - shameless plug ;)
Make sure to subscribe. I intend to blog more regularly about machine learning engineering topics.
Podcasts
[ Machine Learning - SE Daily]
[The O'Reilly Data Show]
For theoretical background
What does interviews look like ?
As it seems most companies will interview you the same way they interview other software engineers. In addition, you may be interviewed on your machine learning theoretical knowledge, data engineering and ML product design.
You can get more details by reading Xiaohan Zeng interviewing experience as a Machine Learning Engineer.
Where can I find suitable opportunities ?
Most of the big companies including the big five have their own machine learning and data teams. It should be easy to find suitable opportunities given that you have the right experience. You can always start from Linkedin.
For Egypt, I found this amazing list of companies that do data and AI.
AI Driven Companies in Egypt
After graduation, I started searching for a good job in the area of Artificial intelligence, specially machine learning, ranking systems, recommendation systems and search engines. It was a hard long journey for me; because I didn't know anything about the Egyptian market in these fields, so I se...
Wrap Up
I tried to give some answers for the what, where and how questions about machine learning engineering positions. It would be nice if you can share your experience getting into the field. As always thanks for reading !
Did you like what you read?
Don't forget to subscribe & please check my other posts on medium and my personal blog
Please share this post so other people can see it which encourage me to write more high quality content!