Real-world impact: Highlights of Le Wagon’s Data Engineering projects

Le Wagon’s Data Engineering bootcamp alumni have showcased their impressive skills through a range of innovative projects. These projects not only highlight their technical knowledge but also their ability to solve real-world problems with data-driven solutions.

Project 1: Restaurant Data Analysis

With this project, students wanted to help restaurants track performance and provide waiters insights into how they can maximise their tips! The team created a dashboard that uses both historical and dynamic data to provide insights into restaurant metrics. They utilised SQL, DBT, and Python to model, transform, and analyse data, incorporating weather information to understand its impact on sales. Two dashboards were developed: one for tracking restaurant profitability and another for waiter performance, offering insights on sales and tips.

Students: Nicolas Jonckheere, Doha Kaddaf, Maria Kosyuchenko, Apolline Jauffret.

Project 2: LinkedIn job market trend analysis

In today’s rapidly changing job market, upskilling and career pivots are essential. This project aimed to help job seekers identify the key skills needed to thrive in a data-driven world. By analysing UK job postings, the team provided valuable insights into market trends. They cleaned and organised the data to create a clear picture of the current job landscape. Their user-friendly dashboard helps job seekers understand which skills are in demand, enabling them to make informed career decisions and stay ahead in their field.

Students: Sifat Khan, Alexandre Valentin, Alexandre Canacaris, Wadiae Lakhlili.

Project 3: Neobank transactions pipeline

Banking is evolving with new technology and user habits. Team NeoBank Pipeline aimed to automate insights for their business units to boost profits.

This project focused on creating a scalable data pipeline for a neobank, managing 2.7 million unorganised transactions. The team built an interactive dashboard that extracts, loads, and models data efficiently. They used DBT for transformations, creating bronze, silver, and gold data layers. The dashboard, deployed on Streamlit, features tailored sections for finance and marketing teams. Future improvements include more automation, schema creation, and enhanced capabilities for continuous integration and delivery.

Students: Alexander Halenke, Luiggi Navilys, Ikechi Ochulo, Marlin Akhter, Enrico Dainelli.

Project 4: Electric Energy Breakdown Analysis

We all strive for a greener world, but what does our energy mix look like at a country level? Team Electric Energy Breakdown is exploring up-to-date global energy consumption and production.

This project visualised energy production and consumption by source and country using time series charts. The team extracted data from an API with a Python script, stored it in a bucket, and transformed it with Google BigQuery and DBT. They used Google Cloud Composer and Airflow for orchestration and scheduling. The final dashboard in Tableau provides personalised views, revealing insights such as Portugal’s lack of production data.

Students: Verdiana Meloni, Desiree Petrilli, Juan Pablo Merea Otermin, Gizem Yilmaz, Konstantinos Panidis

Project 5: U.S. Weather Alerts Pipeline

Planning your next holiday to the US and want to know if it’s the right time? Team U.S. Weather Alerts Pipeline aims to make it easy to check the weather.

This project focused on building a dashboard displaying active US weather alerts. Using the National Weather Service API, the team gathered real-time data to provide essential weather information. They created a Looker Studio dashboard featuring a map of affected counties, alert details, and location-based filtering. The process involved defining the MVP dashboard through user stories and leveraging Apache Airflow and DBT for data collection and transformation. The data flow included bronze, silver, and gold layers for efficient processing and visualisation. The pipeline architecture ensured real-time data storage in Google Cloud Storage, with DBT enhancing data quality and adding geographic context. The result is a seamless transition from raw data to a comprehensive dataset, enabling advanced analysis and visualisation.

Students: Malte Berneaud-Kötz, Kevin Bojda, Mouad Rabih Senhaji, Janine Windhoff.

Conclusion

These projects are a testament to the practical and impactful education provided by Le Wagon, preparing graduates to excel in the data engineering field and drive innovation in their respective industries. They reflect the diverse applications of data engineering across various sectors, showcasing the broad impact that skilled data engineers can have.

If you’re inspired by these projects and want to level up your tech career, consider joining Le Wagon’s upcoming Data Engineering bootcamp session. Visit our website to learn more and secure your spot in the next cohort. Empower yourself with the skills to transform data into impactful solutions and drive your career forward.