Published in Better Programming·PinnedMember-onlyAdvanced SQL techniques for beginnersOn a scale from 1 to 10 how good are your data warehousing skills? — Github On a scale from 1 to 10 how good are your data warehousing skills? Want to go above 7/10? This article is for you. Want to get ready for a data analyst job interview asap? This blog post explains some intricate data warehouse SQL techniques in detail. …Sql7 min readSql7 min read
PinnedMember-onlyUnit Tests for SQL Scripts with Dependencies in Dataformand Data warehouse Gitflow pipelines to run it automatically — Do you unit test your data warehouse scripts? I am going to talk about unit tests for complex SQL queries which might consist of multiple operations (actions). I tried it before using BigQuery scripting: SQL Unit Testing in BigQuery? Here is a tutorial. Complete guide for scripting and UDF testing.towardsdatascience.com However, here is another (and probably better) way to do it.Sql6 min readSql6 min read
Published in Towards Data Science·PinnedMember-onlyTraining your ML model using Google AI Platform and Custom Environment containersComplete guide using Tensorflow, Airflow scheduler and Docker — Google AI Platform allows advanced model training using various environments. So it is really easy to train your model with just one command like so: gcloud ai-platform jobs submit training ${JOB_NAME} \ --region $REGION \ --scale-tier=CUSTOM \ --job-dir ${BUCKET}/jobs/${JOB_NAME} \ --module-name trainer.task \ --package-path trainer \ --config trainer/config/config_train.json …AI7 min readAI7 min read
Published in Towards Data Science·PinnedMember-onlyRetention and Daily Active Users Explained.Complete Data Studio guide and BigQuery tutorial for Firebase users, Machine Learning enthusiasts and Marketers. All you wanted to know. Data Studio template included. — Have you ever wondered how to reduce user churn and save money spent on user acquisition? This article is about how to count those users who stay in your App in order to understand what makes them stay. Who is this article for? Marketers who have been tasked to create custom user retention dashboards.Firebase11 min readFirebase11 min read
Published in Better Programming·5 days agoMember-onlyGreat Data Platforms Use Conventional CommitsYet another way to upgrade your data skills — It’s not a secret that source control is pretty much a standard these days. …even for Data analysts who store their data transformation scripts in repositories. This is not always true though, or there is a better way of doing it. Conventional commits is a great way to improve Github…Big Data5 min readBig Data5 min read
Published in Geek Culture·Jan 29Member-onlyData Warehouse DBA Tasks I Do DailyMonitoring Activity And Managing Resources Like a Pro Or “My table… Where has it gone?” — Often I keep wondering “… Where did that bill come from?” If you want to know everything about what is going on in your data warehouse then this article is for you. There are a few common database administration (DBA) tasks I perform in my data platform daily. Though it’s…Big Data9 min readBig Data9 min read
Published in Towards AI·Jan 28Member-onlyWhen your Stack is a Lake HouseExternal Tables, File Formats, Storage Costs, and Other considerations — It’s never just a data warehouse … yet another way to improve the data platform. If your architecture requires a data lake, then this article is for you. Code snippets you will find below explain how to work with data in AVRO, Parquet , ORC, or JSON and create externally…Data Engineering16 min readData Engineering16 min read
Published in Level Up Coding·Jan 17Member-onlyInfrastructure as Code for BeginnersDeploy Data Pipelines like a pro with these templates — Consider this article as a user-friendly introduction to Infrastructure as Code with a collection of stack file samples to deploy resources that your data platform might need. Infrastructure as code is becoming an increasingly popular approach for managing data platform resources. …Data Engineer11 min readData Engineer11 min read
Published in Towards Data Science·Jan 2Member-onlyData pipeline design patternsChoosing the right architecture with examples — Typically data is processed, extracted, and transformed in steps. Therefore, a sequence of data processing stages can be referred to as a data pipeline. Which design pattern to choose? There are lots of things to consider, i.e. Which data stack to use? What tools to consider? How to design a…Data Engineering9 min readData Engineering9 min read
Published in Towards Data Science·Mar 23, 2022Member-onlyDeploy Machine learning models with Node.js Swagger, BigQuery and AWS CloudformationTutorial how to deploy your machine learning models with just one command — Really simple and easy to learn for beginners. Repository with code can be found here. Outline In this post I will create a simple API and deploy it with AWS Cloudformation. I want to achieve the following: create a Node.JS API to serve my machine learning models. connect API service to a data warehouse solution (in my case it…Nodejs6 min readNodejs6 min read
