🚀Entering Tech #65: How to become a data engineer

Businesses need large amounts of data to make informed decisions. This data is often available only in raw formats and is hard to make sense of. 

Data engineers collect this unprocessed data and turn it into actionable and usable information for data scientists, data analysts, and business analysts, who use it to make business decisions. Data engineers are the unsung heroes, as they make it easier for data scientists and analysts to earn a living. 

Data engineers wear many hats! They source datasets relevant to business goals, develop algorithms to transform raw data into insights, design and maintain data pipelines, collaborate with management to understand needs, create new data validation methods and analysis tools, and ensure everything complies with data governance and security policies.

Too many tasks, you might say; remember, they are heroes, right? 

Before we go further, let’s tell the story of Adeolu Adegboye, who leveraged his data science background to transition into data engineering. Adeolu started learning data science when he thought the world was ending during the COVID-19 lockdown. In 2021, he got an internship as a data scientist and then transitioned to data engineering in 2022 because engineering had always winked at him.

*Newsletter continues after this ad.

Get more from your salary with Eazipay

‘

Join 150,000+ SMEs and employees to get up to 100% extra salaries and more this Workers’ Day! Visit www.myeazipay.com or download the Eazipay Business app to sign up to begin. Contact [email protected] or 07000332947 for more info.

Data engineering is a purely technical field that requires programming knowledge of Python and SQL, along with skills in data modelling, ETL, data management, and data architecture. Adeolu strongly recommends that prospective engineers be versed in the different cloud computing platforms—Amazon Web Services (AWS), Microsoft Azure, and the Google Cloud Platform (GCP).

According to him, data engineering is a holistic skill that requires learning a bit about everything, from database handling to analytical reasoning, machine learning, data security, data storage, and so on.

On the soft skill rung of the ladder, problem-solving, communications and stakeholder management are the most important skills. Data engineers are required to ensure the end user/client understands the full context of what the data provided can do for them. 

If you’re wondering where you ca learn data engineering, Adeolu recommends that you follow the 100-day plan for newbies in data engineering by The Seattle Data Guy. He teaches a range of topics and provides a spreadsheet that houses over 50 courses, challenges, and materials. Adeolu claims that if followed religiously, the 100-day plan could potentially land you an internship or even a full time role if you’re a badass. 

Data professionals I have spoken with for this series say Data Camp is a safe haven, if not heaven, for data professionals. Courses on the platform are made by data professionals and cater to different phases of your learning journey. While courses on DataCamp are priced, Adeolu recommends free alternatives on Codecademy and Coursera. 

Below are some of his recommendations. 

*Newsletter continues after this ad.

The 3i Africa Summit!!!

🚀 Are you ready to be part of the FinTech revolution in Africa and across the globe? Join us, as we prepare for the unforgettable 3i Africa Summit that unites industry leaders, businesses, investors and innovators.

Venue: Accra International Conference Center, Accra, Ghana.

Date: 13th – 15th of May, 2024.

💥Register here to save your spot!! You do not want to miss out on this!!

Sign up for The Big Daily

Ready to get smarter or just hoping to sound like the smartest person in the room at your next networking event? Subscribe to The Big Daily newsletter to get filled in on the day’s most important happenings. Subscribe now → thebigdaily.substack.com.