Data Warehousing with BigQuery: Storage Design, Query Optimisation, and Administration
Looking to learn everything you need to know about the internal workings of BigQuery? This course covers best practices for designing, optimizing, and administering your data warehouse.
Book this courseCourse Overview
3 day course
Certificate of Attendance
Private
Partner of the Year
On this three-day course, you’ll master BigQuery architecture and discover how to design optimal storage and schemas for data ingestion and changes.
Jellyfish has recently been named a Google Cloud Partner. This title recognises our commitment to provide world-leading Cloud-based Training solutions that help our clients succeed. All of our trainers are experienced practitioners, so you can learn with total confidence.
Through a combination of lectures, demos, and labs, you’ll cover techniques to improve read performance, optimise queries, manage workloads, and use logging and monitoring tools.
You’ll also learn about the different pricing models. Finally, you’ll go over various methods to secure data, automate workloads, and build machine learning models with BigQuery ML.
Our Data Warehousing with BigQuery: Storage Design, Query Optimisation and Administration course is available as a private training session that can be delivered live online, at our training centre in The Shard, or at a location of your choice in the UK.
What you'll learn
By the end of this course, you will be able to:
Describe BigQuery architecture fundamentals
Use DML and schedule data transfers to ingest data
Manage capacity and automate workloads
Use logging and monitoring tools to understand and optimise usage patterns
Build and deploy several categories of machine learning models with BigQuery ML
Implement storage and schema design patterns to improve performance
Apply best practices to improve read efficiency and optimise query performance
Understand patterns versus anti-patterns to optimise queries and improve read performance
Apply security best practices to govern data and resources
Course agenda
Module 1: BigQuery Architecture Fundamentals
- Introduction
- BigQuery Core Infrastructure
- BigQuery storage
- BigQuery query processing
- BigQuery data shuffling
Module 2: Storage & Schema Optimisations
- BigQuery storage
- Partitioning and clustering
- Nested and repeated fields
- ARRAY and STRUCT syntax
- Best practices
Module 3: Ingesting Data
- Data ingestion options
- Batch ingestion
- Streaming ingestion
- Legacy streaming API
- BigQuery storage write API
- Query materialisation
- Query external data sources
- Data transfer service
Module 4: Changing Data
- Managing change in data warehouses
- Handling slowly changing dimensions (SCD)
- DML statements
- DML best practices and common issues
Module 5: Improving Read Performance
- BigQuery's cache
- Materialised views
- BI engine
- High throughput reads
- BigQuery storage read API
Module 6: Optimising & Troubleshooting Queries
- Simple query execution
- SELECTs and aggregation
- JOINs and skewed JOINs
- Filtering and ordering
- Best practices for functions
Module 7: Workload Management & Pricing
- BigQuery slots
- Pricing models and estimates
- Slot reservations
- Controlling costs
Module 8: Logging & Monitoring
- Cloud monitoring
- BigQuery admin panel
- Cloud audit logs
- INFORMATION_SCHEMA
- Query path and common errors
Module 9: Security in BigQuery
- Secure resources with IAM
- Authorised views
- Secure data with classification
- Encryption
- Data discovery and governance
Module 10: Automating Workloads
- Scheduling queries
- Scripting
- Stored procedures
- Integration with Big Data products
Module 11: Machine Learning in BigQuery
- Introduction to BigQuery ML
- How to make predictions with BigQuery ML
- How to build and deploy a recommendation system with BigQuery ML
- How to build and deploy a demand forecasting solution with BigQuery ML
- Time-series model with BigQuery ML
- BigQuery ML explainability
Who it's for
This course is ideal for data analysts, data scientists, data engineers, and developers who perform work on a scale that requires advanced BigQuery internal knowledge to optimise performance.
Prerequisites
To get the most out of this course, you should have completed the Google Cloud Fundamentals: Big Data and Machine Learning course.
BOOK THIS COURSE
Enquire for a team or large group
For private sessions call our sales team