Tutorial how to deploy your machine learning models with just one command — Really simple and easy to learn for beginners. Repository with code can be found here. Outline In this post I will create a simple API and deploy it with AWS Cloudformation. I want to achieve the following: create a Node.JS API to serve my machine learning models. connect API service to a data warehouse solution (in my case it…

Data warehouse guide for better and cleaner data with scheduled emails — Building a data warehouse with clean data If you are building a data warehouse solution or/and running some admin tasks in databases then this article is for you. It answers this question: How to test data quality and send email notifications in case something is not quite right. Ideally every data user would like to be notified…

How to build one and export millions of rows in chunks, stream, capture real-time data changes or extract data and save it to the Cloud — Building MySQL data connector for your data warehouse Imagine you are a Data engineer and you were tasked to sync data from one of your MySQL database instances and your data warehouse. This is a very common scenario in which you need to connect of the most popular data sources and your data warehouse solution, i.e. Snowflake, Redshift…

File formats, yaml pipe definitions, and transform and event triggers for your simple and reliable data ingestion manager — - “You can now simply load all the files from Cloud Storage which match your pipe definition with just one ‘$ npm run test’ command or invoke data loading when files are being created there. Github repository with code

And always have an up-to-date data for your custom reports — If you are a Firebase or Google Analytics 4 user and you have setup data imports into your BigQuery data warehouse then you might want to create real-time custom reports with your data in intraday schema. The problem is that this integrated dataset is being deleted automatically by Google every…

Or from any arbitrary data source API using Serverless Node.js and perform any ETL/ELT before loading into your data warehouse — Github repository with code Project outline You will find how to: Create a PayPal account with developer access and a Sandbox and mock test transactions Create a sandbox account to integrate and test code in PayPal’s testing environment. Populate your testing environment with some transaction data

And start with data graphs for SQL transformation using Dataform — Let’s say you are building your Data Warehouse solution using BigQuery, Snowflake, Redshit, etc. You were tasked to create a few reports and you need to create production and test environments for SQL transformations you do daily. You would want to run data transformations as code too pushing changes to…

Complete guide using Tensorflow, Airflow scheduler and Docker — Google AI Platform allows advanced model training using various environments. So it is really easy to train your model with just one command like so: gcloud ai-platform jobs submit training ${JOB_NAME} \ --region $REGION \ --scale-tier=CUSTOM \ --job-dir ${BUCKET}/jobs/${JOB_NAME} \ --module-name trainer.task \ --package-path trainer \ --config trainer/config/config_train.json …

Why I don’t trust User Activity Dashboards — A complete SQL guide for marketers and machine learning engineers. MAU, DAU and WAU. Firebase and BigQuery example. Nifty report template included. Read how to copy it in the end of this article. It’s free! Counting Active users in your App might be tricky. What would you do? …