BigData Engineer | Full stack dev | I write about ML/AI in Digital marketing. | linktr.ee/mshakhomirov | @MShakhomirov

Complete Data Studio guide and BigQuery tutorial for Firebase users, Machine Learning enthusiasts and Marketers. All you wanted to know. Data Studio template included.

Data Studio template. Image by author

Have you ever wondered how to reduce user churn and save money spent on user acquisition? This article is about how to count those users who stay in your App in order to understand what makes them stay.

Who is this article for?

  • Marketers who have been tasked to create custom user retention dashboards.
  • Analysts…

How to build one and export millions of rows in chunks, stream, capture real-time data changes or extract data and save it to the Cloud

Photo by Rita Seneca on Unsplash

Building MySQL data connector for your data warehouse

Imagine you are a Data engineer and you were tasked to sync data from one of your MySQL database instances and your data warehouse. This is a very common scenario in which you need to connect of the most popular data sources and your data warehouse solution, i.e. …

Hands-on Tutorials

File formats, yaml pipe definitions, and transform and event triggers for your simple and reliable data ingestion manager

Sample pipe definition. Image by Author 💡Mike Shakhomirov

- “You can now simply load all the files from Cloud Storage which match your pipe definition with just one ‘$ npm run test’ command or invoke data loading when files are being created there.

Github repository with code

And always have an up-to-date data for your custom reports

Photo by Jessica Lewis on Unsplash

If you are a Firebase or Google Analytics 4 user and you have setup data imports into your BigQuery data warehouse then you might want to create real-time custom reports with your data in intraday schema. The problem is that this integrated dataset is being deleted automatically by Google every…

Or from any arbitrary data source API using Serverless Node.js and perform any ETL/ELT before loading into your data warehouse

Image by author

Github repository with code

Project outline

You will find how to:

Create a PayPal account with developer access and a Sandbox and mock test transactions

  • Create a sandbox account to integrate and test code in PayPal’s testing environment.
  • Populate your testing environment with some transaction data
  • Try to CURL this data from PayPal reporting API.

Create a PayPal data connector with AWS Lambda

  • Connect to your PayPal account and use PayPal API Authorization token to extract transaction data, for example, for yesterday.
  • Use…

And start with data graphs for SQL transformation using Dataform

Dataform dependency graph. Image by author

Let’s say you are building your Data Warehouse solution using BigQuery, Snowflake, Redshit, etc. You were tasked to create a few reports and you need to create production and test environments for SQL transformations you do daily.

You would want to run data transformations as code too pushing changes to…

Complete guide using Tensorflow, Airflow scheduler and Docker

Photo by Setyaki Irham on Unsplash

Google AI Platform allows advanced model training using various environments. So it is really easy to train your model with just one command like so:

gcloud ai-platform jobs submit training ${JOB_NAME} \
--region $REGION \
--scale-tier=CUSTOM \
--job-dir ${BUCKET}/jobs/${JOB_NAME} \
--module-name trainer.task \
--package-path trainer \
--config trainer/config/config_train.json …

Complete guide for scripting and UDF testing

Photo by Florian Olivo on Unsplash

Since Google BigQuery introduced Dynamic SQL it has become a lot easier to run repeating tasks with scripting jobs. Now we can do unit tests for datasets and UDFs in this popular data warehouse.

What is this article about?

This tutorial aims to answers the following questions:

  1. How to write unit tests for SQL and…

Getting Started

Complete Python comparison and Step by Step guide for any dataset. Kaggle User churn data.

Exploratory Data Analysis in Google Data Studio. Image by author.

Can we perform Exploratory Data Analysis with SQL?

— Yes, we can.

What is this article about?

It is about Exploratory Data Analysis (EDA) and aims to answer the following questions:

  • What is Exploratory Data Analysis (EDA)?
  • How to perform Exploratory Data Analysis (EDA) in Pandas (Python)?
  • How to perform Exploratory Data Analysis (EDA) in…

💡Mike Shakhomirov

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store