--- title: Application of Data Engineering Techniques | MIT xPRO --- [![Hubspot_HalfSizeMargin_xPRO_sub-brand_lockup_one-line_cmyk_black](https://learn-xpro.mit.edu/hs-fs/hubfs/Hubspot_HalfSizeMargin_xPRO_sub-brand_lockup_one-line_cmyk_black.png?noresize&width=12719&height=5535&name=Hubspot_HalfSizeMargin_xPRO_sub-brand_lockup_one-line_cmyk_black.png)](https://xpro.mit.edu/) SEE ENROLLMENT INFO BELOW ### SEE ENROLLMENT INFO BELOW # Application of Data Engineering Techniques This 4 week online course will teach you practical skills for how to prepare your data for use in real-world analytics scenarios by using techniques such as advanced data modeling, ETL/ELT, SQL, and data pipelines in GCP. Additionally, you will cover how to use cloud data in Python/R solutions. ###### Date ### Oct 14 - Nov 11, 2024 ###### Duration ### 4 weeks, 5 hours per week ###### PRICE ### $1,250     ###### FUTURE DATES ### TBD ###### CEUs ### 2.0 CEUs Description Goes Here ###### Course Number ### ADETxB Description Goes Here ### TOPICS COVERED - Advanced SQL techniques - Advanced data processing techniques in R/Python - Working with real-time data - Introduction to Google Pub/Sub - Streaming data into BigQuery - ETL processing pipelines - Data cleaning – GCP based techniques for cleaning data - Python/R integration with the Google data management environment and data manipulation - Advanced data cleansing using pipelines - Data pipelines and automation, Airflow, Config and management - Case study, a real world example of building a data pipeline ## Why this training is relevant ### Discover how data engineering skills can unlock the potential of generative AI: - **Data Pipeline Management: **Data engineers build and maintain pipelines for collecting, cleaning, and preparing large volumes of data crucial for training generative AI models. - **Data Storage and Management: **Data engineers are responsible for developing systems to store and manage data generated by generative AI models. - **Quality Improvement of Existing Data: **Generative AI models, under data engineers' guidance, fill in missing data, correct errors, and generate new features in existing datasets. - **Efficiency and Scalability: **Data engineers leverage data processing and distributed computing knowledge. Knowledge and skills on AI help engineers design and implement efficient and scalable data pipelines and computing systems for training and running generative AI models. - **Reliability and Robustness: **Data engineers ensure high-quality data through knowledge of data quality and governance. Data engineer skills are required to train and run generative AI models in a reliable and robust manner. - **Accessibility and User-Friendly Models: **Data engineers use software engineering and user experience design. These skills are requiered to develop tools and platforms for easy training, running, and deployment of generative AI models. ### TARGET AUDIENCE - Data Engineers - Data Architects - Data Scientists - Data Analysts - IT staff, & anyone who needs to work with and structure data for use for analysis, reporting, machine learning or applications   ![Image](https://learn-xpro.mit.edu/hubfs/assets/images/placeholders/600x400.png) ### PREREQUISITES - Knowledge of how to engineer foundational data solutions - Knowledge of the various types of data storage and data management techniques that are available in Google Cloud - Understand the modern data stack and how to set up pipelines for data engineering automation - Understand basic data transformation techniques - Knowledge of SQL for basic data engineering operations ## LEARNING OUTCOMES Understand how to engineer advanced data solutions, including setting up Cloud-based data pipelines. Understand and manage different types of advanced databases (streaming, NoSQL, scalable) and examine when to use the most appropriate type for each situation.  Understand how to design and implement data pipelines, DevOps, and data transformation methods. Understand advanced structuring and data querying. Manage data and apply advanced visualization techniques. ## OTHER CLOUD DATA ANALYTICS TRANSFORMATION COURSES Upon successful completion of all "Fundamentals of Data" courses, you will be rewarded with a program certificate on Cloud Data Analytics Transformation, in addition to each individual course certificate. | #### [Application of Data Visualization](https://learn-xpro.mit.edu/advt_boeing) [![DVCA](https://learn-xpro.mit.edu/hs-fs/hubfs/DVCA.png?width=432&height=243&name=DVCA.png)](https://learn-xpro.mit.edu/advt_boeing) | #### [Application of Data Science](https://learn-xpro.mit.edu/adst_boeing) [![DSCA](https://learn-xpro.mit.edu/hs-fs/hubfs/DSCA.png?width=432&height=243&name=DSCA.png)](https://learn-xpro.mit.edu/adst_boeing) | | --- | --- | | #### [Fundamentals of Data Visualization](https://learn-xpro.mit.edu/fdv_boeing) [![Woman working on a dashboard](https://learn-xpro.mit.edu/hubfs/DVCB.jpeg)](https://learn-xpro.mit.edu/fdv_boeing) | #### [Fundamentals of Data Engineering](https://learn-xpro.mit.edu/fde_boeing) [![DECB](https://learn-xpro.mit.edu/hs-fs/hubfs/DECB.jpeg?width=431&height=242&name=DECB.jpeg)](https://learn-xpro.mit.edu/fde_boeing) | #### [Fundamentals of Data Science](https://learn-xpro.mit.edu/fds_boeing) [![Two man working on data visualization dashboards ](https://learn-xpro.mit.edu/hubfs/DSCB.jpeg)](https://learn-xpro.mit.edu/fds_boeing) | | --- | --- | --- | ## MIT INSTRUCTORS ![Cloud_John_Williams](https://learn-xpro.mit.edu/hubfs/Data%20Science%20/Cloud%20Data/Cloud_John_Williams.jpeg) ### John Williams Professor of Information Engineering and Civil and Environmental Engineering at MIT ![Cloud_Sanchez](https://learn-xpro.mit.edu/hubfs/Data%20Science%20/Cloud%20Data/Cloud_Sanchez.jpeg) ### Abel Sanchez Executive Director, Research Scientist, Laboratory for Manufacturing and Productivity, MIT ## The MIT xPRO Learning Experience We bring together an innovative pedagogy paired with world class faculty. ![](https://learn-xpro.mit.edu/hs-fs/hubfs/assets/images/placeholders/icon2.png?width=175&height=110&name=icon2.png) ### Learn by doing Practice process and methods through labs, simulations, assessments, case studies, and tools.  ![](https://learn-xpro.mit.edu/hs-fs/hubfs/assets/images/placeholders/icon6.png?width=145&height=110&name=icon6.png) ### Learn from others Connect with an international community of professionals while working on projects based on real-world examples. ![](https://learn-xpro.mit.edu/hs-fs/hubfs/assets/images/placeholders/Work_Life_Balance.png?width=161&height=107&name=Work_Life_Balance.png) ### Learn on demand Access all of the content online and watch videos on the go. ![](https://learn-xpro.mit.edu/hs-fs/hubfs/assets/images/placeholders/icon3.png?width=149&height=110&name=icon3.png) ### Reflect and Apply Bring your new skills to your organization, through examples from technical work environments and ample prompts for reflection.   ![](https://learn-xpro.mit.edu/hs-fs/hubfs/assets/images/placeholders/icon5.png?width=121&height=109&name=icon5.png) ### Demonstrate your success Earn a Professional Certificate from MIT xPRO and Continuing Education Units (CEUs). ![](https://learn-xpro.mit.edu/hs-fs/hubfs/assets/images/placeholders/icon4.png?width=113&height=110&name=icon4.png) ### Learn from the best Gain insights from MIT faculty and industry experts and connect with your course team and faculty over discussion forums and live virtual office hours.   ## HOW TO ENROLL **Boeing employees:** All courses on this page are eligible for the Learning Together Program (LTP). Please check your eligibility to participate and then follow these instructions to enroll. Note: All vouchers must be submitted 5 business days prior to the course start date. 1. Review LTP policy and process details on Worklife > Browse Menu > Career > Learning Together Program. 2. Identify the MIT xPRO course(s) you would like to enroll in. 3. Courses may be added to the LTP system within 60 days of the course start date (U.S.), or up to one year in advance (ILHE – International Locally Hired Employee). 4. Drop Policy: Course refunds are given (no questions asked) through the first Monday of each course. You must cancel your course in the LTP system as well as with MIT xPRO. Follow instructions on Worklife to cancel in LTP system. To drop with MIT, please email [support@xpro.mit.edu](mailto:support@xpro.mit.edu). 5. Go to My LTP from LTP Worklife portal. Select Massachusetts Institute of Technology as the school for enrollment. Complete all required fields on the LTP online enrollment. 6. U.S. employees will populate a tuition Voucher for billing purposes for each course by way of the LTP enrollment. 7. Employees outside the U.S. will populate a Sponsorship Letter for billing purposes. **Please note:** - **International Locally Hired Employees (ILHEs) must personally pay upfront (corp card is not permitted) and request reimbursement from LTP.** - **Effective October 1, 2023, ILHEs from India will be asked to add 18% to the tuition price representing the tax levy that India is imposing on all online educational programs from US universities.** 8. Create an account at [xpro.mit.edu](http://xpro.mit.edu/) or login with an existing account. Use one of the supported browsers: Chrome, Safari or Firefox.  9. U.S. employees must upload the Boeing LTP Voucher to [xpro.mit.edu/boeing/upload](http://xpro.mit.edu/boeing/upload) to authorize payment to be made to school. After you successfully upload your Voucher, you will be asked to confirm your course selection. 10. ILHE participants must complete this [form](https://mit.co1.qualtrics.com/jfe/form/SV_1YceWO0xItBevnn) and upload along with your Sponsorship Letter. Note: ILHE participants will need to upload the school's document of tuition cost for each course and the local payment center will process reimbursement. 11. You will need to submit your certificate of completion to the LTP system to confirm grade requirements are met and to close out each course. **Important Note**: - If errors are found on the Vouchers -- for example, the course start date is incorrect -- MIT xPRO will identify the error and request that you resubmit the Voucher with the correct information.   If you have any questions about the enrollment process or if your Voucher is not accepted, please visit the [MIT xPRO Support Center](https://xpro.zendesk.com/hc/en-us/requests/new?__hstc=205621196.51221cc9801d122cf951c192ddaa8517.1619873736599.1620403340421.1620416951216.9&__hssc=205621196.20.1620416951216&__hsfp=391747837). For LTP-related questions, call or submit an electronic ticket via Worklife > Get Support. [![MIT](https://learn-xpro.mit.edu/hubfs/mit_logo_std_rgb_white.png)](https://web.mit.edu/) Massachusetts Institute of Technology 77 Massachusetts Avenue Cambridge, MA 02139 [© 2026 All Rights Reserved. MIT xPRO.](https://mitxpro.mit.edu/)