(image by @mamihery) Why Hy? My second programming language was a Lisp. I fell in love with the expressiveness of Lisps. But most Lisps are obscure. The ecosystem is not suited for beginners. One of younger Lisps, Clojure, was my first foray into the world of web-development. I failed spectacularly because I didn’t know how web-develoment worked, and the Clojure ecosystem assumes too much previous knowledge. Other languages, e.g., Racket, are more approachable, but are still too unpopular to have a big community.

Here are some links for this Friday: Architecture Patterns with Python: Enabling Test-Driven Development, Domain-Driven Design, and Event-Driven Microservices Paperback - learn Domain-Driven Design with Python, free HTML version Performance Best Practices: Running and Monitoring Express.js in Production - insightful article, although it boasts some ads for the service the blog belongs to How To Build An Amazing LinkedIn Profile [15+ proven tips] - a guide on getting the most out of your LinkedIn profile Nonsense!

You can get two Lisp-related books for free on Leanpub: Lisp Hackers Programming Algorithms There’s also a book on the Hy Programming language, a Lisp-like syntax alternative for Python. The book A Lisp Programmer Living in Python-Land: The Hy Programming Language is not free, but available for a minimum price of $ 5.00. It contains some practical projects for Hy, for example, web-scraping with the library Beautiful Soup, or Deep Learning with Keras and Tensorflow.

Streamlit allows you to write Markdown within a Python file (.py): import streamlit as st st.title("Otto Group Product Classification Challenge 🛍") st.markdown("## 1. Problem Statement") st.markdown( "Given a [dataset](https://www.kaggle.com/c/otto-group-product-classification-challenge/overview) with **93 features**, create a **predictive model** which is able to **distinguish between the main product categories**." ) st.markdown("### 1.2 Evaluation") st.markdown( "The evaluation for the competition is **multi-class logarithm loss**. See [Kaggle: Evaluation.](https://www.kaggle.com/c/otto-group-product-classification-challenge/overview/evaluation)" ) I like that I can write Markdown, but the syntax is cumbersome.

Create a Docker container that runs your machine learning models as a web application This article will explain the advantages of Streamlit and how to build a Streamlit application with Docker. Why Streamlit? You’ve explored your data and developed a machine learning model. It’s now time to release it to the world so that others can see what you’ve built. Now what? Deploying machine learning models is not trivial.

I finished the Complete Machine Learning and Data Science: Zero to Mastery this weekend (and wrote about it). The course has given me the foundations of working with data in Python. Practice makes perfect. My goal is to sharpen my skills by exploring a Kaggle dataset, building a model and deploying it with Streamlit using Docker and Heroku. The project will be on GitHub where I will post all the links, my thoughts and observations.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the eigth part of the blog post series. part 1 part 2 part 3 part 4 part 5 part 6 part 7 TL;DR (A Review of The Complete Course) The program is a praise-worthy introduction to data science and machine learning with Python. The instructors focus on practical skills and convey an enormous topic in a captivating and friendly way.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the seventh part of the blog post series. part 1 part 2 part 3 part 4 part 5 part 6 part 8 13. Data Engineering These lectures cover what kind of data we have (structured data, unstructured data, etc.). How can we make the raw data consumable for machine learning libraries?

Create a pipeline to score different machine learning models with scikit-learn After the initial data exploration I would like to get a quick gauge on what model would be best for the problem at hand. A rough estimate helps in narrowing which machine-learning model to use and tune later. It helps to get a sense on how effective perspective algorithms will be. The goal is to get a big picture overview.

Google Colab is a free online coding environment that offers GPU acceleration for your data science and machine learning needs. It runs on top of Jupyter Notebooks. That means that the interface is familiar to most data scientists that use Python. If your local machine is too slow for some of the more intensive computations you need for machine learning, Colab can help you out. When you use the remote runtime with the free GPU, the runtime disconnects after a while.

You want to score a list of models with cross-validation with customized scoring methods. Why Cross-validation? A common approach to machine learning is to split your data into three different sets: a training set, a test set, and a validation set. Learning the parameters of a prediction function and testing it on the same data is a methodological mistake: a model that would just repeat the labels of the samples that it has just seen would have a perfect score but would fail to predict anything useful on yet-unseen data.

Google Colab is an online tool that allows you to run Python notebooks with free GPU acceleration. Why is that useful? Some machine learning models take a long time to compute and your local machine might not be able to run them. The Colab notebooks are similar to Jupyter Notebooks, but they use the Google Drive environment. You can always upload your dataset to Google Drive and connect your Drive to Colab.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the sixth part of the blog post series. part 1 part 2 part 3 part 4 part 5 part 7 part 8 10. Milestone Project 1 In this project we work through a dataset from start to finish. We use supervised machine learning to gain insight into a classification problem.

As I’m working through different data sets in my machine learning journey, it gets more obvious that you have to know about feature engineering. Feature engineering is an umbrella term for transforming your input data. A machine learning model can only be as efficient as the data you feed it. Features are the different input qualities you give the model. Often, these features are in the wrong format, or they are missing.

Today I learned about logistic regression. Logistic Regression is a statistical model that we can use for classification problems in machine learning. You can easily confuse the term with linear regression. With linear regression, you model the probability of a quantitative value, for example a price. With logistic regression you can predict categories: yes/no, pass/fail, etc. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.

I’ve recently begun learning about data science and machine learning. Here are some resources that I found (sorted alphabetically): A Lightning-Fast Introduction To Deep Learning And Tensorflow 2.0 An Introduction to Statistical Learning Awesome Data Science Awesome ML Ops Awesome Machine Learning and AI Courses Awesome Machine Learning Building Machine Learning Pipelines Chris Albon’s Notes Coursera Mathematics for Machine Learning Daniel Bourke’s Resources and his AI Masters Degree Data Science from Scratch: First Principles with Python Deploying and Hosting a Machine Learning Model with FastAPI and Heroku End to End Machine Learning Tutorial — From Data Collection to Deployment 🚀 Full Stack Deep Learning: how to deploy models Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems Khan Academy: Math Learning Math for Machine Learning MIT Deep Learning Book ML Ops ML from the Fundamentals Machine Learning Algorithms from Scratch in Python Machine Learning Engineering book Machine Learning Glossary Machine Learning Mastery: Start Here Machine Learning from Scratch: covers the building blocks of the most common methods in machine learning Mathematics For Machine Learning Mathematics for the adventurous self-learner Neural Networks and Deep Learning Putting ML in Production: a guide and code-driven case study on MLOps for software engineers, data scientists and product managers Python Data Science Handbook Resources for learning numpy, pandas, etc.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the fifth part of the blog post series. part 1 part 2 part 3 part 4 part 6 part 7 part 8 9. Scikit-Learn Up until now, we’ve learned how to consume data and make fancy diagrams. The current section finally deals with Machine Learning and teaches you the basics of Scikit-learn.

Today I learned how to reduce feature labels in a data set with Principal Component Analysis. From Python Data Science Handbook: Principal component analysis is a fast and flexible unsupervised method for dimensionality reduction in data, […] You can use PCA to learn about the relationship between two values: In principal component analysis, this relationship is quantified by finding a list of the principal axes in the data, and using those axes to describe the dataset.

Today I learned that if you slice a list in Python, the program returns a copy of the list. But NumPy returns a view, not a copy. That means, that slicing a NumPy array will modify it: This default behavior is actually quite useful: it means that when we work with large datasets, we can access and process pieces of these datasets without the need to copy the underlying data buffer.

Let’s say we made some predictions with a machine-learning model using scikit-learn. We want to evaluate how our model performs, and create a confusion matrix: from sklearn.metrics import confusion_matrix ## make predictions with the scikit-learn model on the test data set y_preds = model.predict(X_test) ## Create confusion matrix on test data and predictions cm = confusion_matrix(y_test, y_preds) cm You’ll get an array like this: array([[24, 5], [ 4, 28]]) We can visualize it with pandas:

If you want to convert a CSV file into Pandas, you can use [pandas.read_csv][readcsv]. The function takes several options. One of them is sep (default value is ,). You can use a regular expression to customize the delimiter. Let’s say your data looks like this: vhigh,high,2,2,more,small med,vhigh,3,more,big ... You want to load that data into a Pandas DataFrame. You can split each line on the comma, but you want to ignore the comma inside floating point numbers like 2.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the fourth part of the blog post series. part 1 part 2 part 3 part 5 part 6 part 7 part 8 7. NumPy The section covers an introduction into NumPy. NumPy will covert any data into a series of numbers. NumPy is the backbone of all data-science in Python.

If you want to make JupyterLab more Vim-like, you can use Vim key bindings and install the jupyterlab-vim extension. 1. Vim Key Bindings The notebook UI has the option to use Vim, Emacs, or the default key mappings. The documentation shows how. In my Docker image I create a json file to hard code these settings: File jupyter-codemirror-settings.json: { "keyMap": "vim" } Dockerfile: ## previous setup - base image, working directory, etc.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the third part of the blog post series. part 1 part 2 part 4 part 5 part 6 part 7 part 8 4. The 2 Paths The class aims to be beginner-friendly. Now you have the choice to learn how to program in Python or to continue with the default route.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery and writing down my observations/lecture notes. This is the second part of the blog post series. part 1 part 3 part 4 part 5 part 6 part 7 part 8 3. Machine Learning and Data Science Framework The course focusses on learning by doing. Instead of learning higher mathematics and over-thinking the process, the instructors show you a framework that encourages a fast feedback loop.

I’m going through the Udemy course Complete Machine Learning and Data Science: Zero to Mastery. The course runs under the flag of Andrei Neagoie. Andrei is a popular instructor on Udemy, with almost 200.000 students, and top reviews. For this course, he has paired up with Daniel Bourke, a self-taught Machine Learning Engineer from Australia. In this blog post series, I will jot down my thoughts on the course, and what I’ve learned.

Here are 5 links for this Friday: How to create a simple REST API with Python and Flask in 5 minutes - a beginner-friendly Python & Flask tutorial What is Phoenix LiveView - a good overview about the new experimental feature in Elixir’s web framework For a Good strftime - if you quickly want to look up how to format a string in Python Introduction to CSS - Youtube series about CSS A Recap of Frontend Development in 2019 - what happened in 2019?

Today I learned about the ASGI specification for Python frameworks: ASGI (Asynchronous Server Gateway Interface) is a spiritual successor to WSGI, intended to provide a standard interface between async-capable Python web servers, frameworks, and applications. Python didn’t have asynchronous execution until recently. Since Python 3.4 there’s asyncio in the standard library. asyncio is singl-threaded, single-process - like JavaScript’s event loop. ASGI is build on top of asyncio. So, basically, Python now has web servers and frameworks with async capacities.

I’m currently creating some micro-service applications with Docker and Flask. Python is quite a beginner-friendly language, and Flask is easy to get started with. At the same time, it isn’t straightforward to add more than basic features. For example, the tutorials use Flask-RESTful. Adding Swagger-UI to my “standard” Docker image is quite a hassle. You can use flask-swagger-ui, but then you have to figure out how to serve the static json file.

A few days ago, I created a Docker build for Flask with PostgreSQL (both with Alpine Linux and with Debian Linux). Installing psypcopg-2 binary (required for Postgres) requires you to build the package from source. Now the Docker image grows in size, as it still contains the build artifacts. The solution? Multi-stage Docker builds. Let’s say we have the following docker-compose.yml file. There are two services: a Flask API called users and a Postgres database called users-db.

Create isolated Python projects with virtual environments What is a virtual environments and why should I use it? A virtual environment allows you to develop several Python projects with different versions of packages on the same computer. Python usually installs the latest versions of your dependencies globally. You’ll run into problems, if one of your projects requires a different package version. venv Python ships with venv out of the box since version 3.

How to build a Python app with PostgreSQL I’m currently setting up a Flask app with PostgreSQL and Docker. Like most examples you’ll find on the internet, the course I’m following uses Alpine Linux as a base image. Alpine’s selling point is the small image size. But Alpine uses a different C library, musl, instead of glibc. That’s one of the reasons why the website Pythonspeed recommends Debian Buster as the base image for Python (as of 2019).

Here are some sweet Black Friday Deals. Wes Bos Wes Bos creates high-quality courses about JavaScript and React. Now all of those courses are on sale. I bought the FullStack Advanced React & GraphQL course. Wes glosses over some concepts, but overall the quality is quite high, and the course was well worth the money. Find an overview over all courses at wesbos.com/courses. Talk Python The Mega-Bundle Deal is $50 off: $149 for the whole course catalog.

I started the Microservices with Docker, Flask, and React a while ago. The course proved to be quite challenging, as I’m not familiar with Docker and Flask. It also uses advanced features of React. It was very fascinating for me to see how to develop a modern application with TDD, using Docker. While the React part is a bit outdated (it still uses deprecated methods like getDerivedState) - everything works and teaches you useful concepts.

Here are some things I’ve found useful this week or that I enjoyed: A web scraping guide for beginners, part 1 and part 2 A weird experiment: The Boost Your Coding Fu With VSCode and Vim Auditory Experience! An Audiobook? A Podcast?! Just Enough Erlang Truly reactive programming with Svelte 3.0 - this post shows what’s exciting about Svelte.js 4 Rules for Intuitive UX - the minimum to get you started - a must read!

I’m learning test-driven development with the course Microservices with Docker, Flask, and React. It’s been a lot of fun. I’ve also learned more about using Docker and docker-compose. The course uses Coverage.py for measuring Python code coverage. However, Coverage.py and Docker don’t play well with each other if you run the Docker container as a normal (non-root) user. PermissionError: [Errno 13] Permission denied: 'usr/src/app/.coverage' Here is the Dockerfile: ## base image FROM python:3.

I’m interested in learning test-driven development (and Domain-Driven Design, too). I find it a bit hard to find the right material to apply TDD in the context of building a full-stack web application. I’m familiar with building Node.js applications, and have a passing knowledge of NestJS. So far, the best material about TDD for back-end development seems to come with Python - either Flask or Django. While I’m not interested in learning yet another web framework, it seems that learning TDD for beginners comes with Python.

Today I reviewed my progress with the Microservices with Docker, Flask, and React by Michael Herman. I’m not familiar with Flask, but I wanted to learn how to build a microservices app with TDD. The testdriven.io course was the best course I could find which uses Docker, and React and teaches testing principles. The parts about Docker proved already useful. I used the setup to dockerize a simple Django app.

Here are some things I’ve found useful this week or that I enjoyed: The Architecture of Open Source Applications - some free books on architecture! Use Svelte with SASS/SCSS in VSCode - add SASS to your Svelte project An Introduction to Domain-Driven Design - DDD w/ TypeScript - provides a good overview on DDD How to make your NodeJS application or API secure - some beginner-friendly advice on how to tackle security Mastering Phoenix Framework - free online version of a guide to Phoenix with TDD Building a JSON API in Elixir with Phoenix 1.

I’m trying to use Docker for my apps. I like the tool, and it also helps to have a stable dev environment when everything runs inside a container. But it adds another layer of complexity. Now, a lot goes wrong with trying to set up Docker, docker-compose, etc. correctly. Luckily, I’m happy to fiddle with tools (would I be using Vim, Colemak, Arch Linux otherwise?). The free preview of the Microservices with Docker, Flask, and React should give you an overview of how to use Docker with Python.

Yesterday I finished the course Build a Backend REST API with Python & Django - Advanced. Here is my review: Excellent course on TDD with Django. I especially like how we use Docker and Travis CI to build a robust dev environment. The instructor is easy to understand. As this is an advanced course, he doesn’t go into too many details. But he provides links to further documentation and encourages you to build your knowledge on your own.

Today I tried to set up a dockerized Flask app with Postgres database with Travis CI. It shouldn’t have been difficult because I followed a tutorial. Setup Here is the docker-compose.yml: version: '3.7' services: users: build: context: ./services/users dockerfile: Dockerfile volumes: - './services/users:/usr/src/app' ports: - 5001:5000 environment: - FLASK_ENV=development - APP_SETTINGS=project.config.DevelopmentConfig - DATABASE_URL=postgres://postgres:postgres@users-db:5432/users_dev - DATABASE_TEST_URL=postgres://postgres:postgres@users-db:5432/users_test depends_on: - users-db users-db: build: context: ./services/users/project/db dockerfile: Dockerfile ports: - 5435:5432 environment: - POSTGRES_USER=postgres - POSTGRES_PASSWORD=postgres nginx: build: context: .

I have several courses, books, and video platforms I learn from. One of my recent discoveries is Testdriven.io, a website dedicated to full-stack web development with Python and TDD. I purchased the Microservices with Docker, Flask, and React course in February. I’m happy to say that the author updates the course regularly. This proves to be invaluable, as outdated courses are a waste of money. You begin by setting up Docker with Python and Flask and I already learned a lot.

I’ve tried several online courses on learning data structures and algorithms. But nothing seems to stick. Usually, video courses work well for me. I consider them more engaging than a book. But for algorithms, it doesn’t work for me. I’ve now bought some titles on learning how to solve algorithm challenges and books on problem-solving: Think Like a Programmer This book teaches you how to solve programming problems systematically.

Here are all falsy values: JavaScript Python 3 Clojure/ClojureScript Falsy false False false null None nil NaN undefined 0 (zero) 0 0.0 0.0 "" (empty string) "" (empty string) [] (empty list) () (empty tuple) {} (empty dict) set() (empty set) b'' (empty byte) Everything else is truthy.