Other articles

1. Pelican on Cloudflare - Generating a Python-based static blog with Cloudflare Pages

   Problem

   Deploy Pelican static site (Python-based static blog) to Cloudflare Pages

   Today I Learnt

   This blog, built using Pelican (Python static site generator), used to be hosted on commons.host for free static site hosting. To my dismay, commons.host is no longer live and I needed a replacement static …

   read more

2. Connecting to Microsoft SQL Server using SQLAlchemy and PyODBC

   Problem

   Connect to a remotely-hosted Microsoft SQL Server within a Python script, using SQLAlchemy as a database abstraction toolkit and PyODBC as a connection engine to access the database within the remotely-hosted SQL Server.

   Today I Learnt

   When writing programs that involve interacting with a database, we need to use …

   read more

3. Getting Started with PySpark

   Some Context

   I have been using the pandas library for almost 2 years now, but I have always been interested in getting started with using PySpark in a big data project. Since I intend to build a daily habit of taking notes of what I've learnt (which I haven't really …

   read more

4. Grid Search Hyperparameter Optimization in Scikit-learn with GridSearchCV

   Problem

   Choose a set of optimal hyperparameters for a machine learning algorithm in scikit-learn by using grid search

   Today I Learnt

   When training a machine learning model, model performance is based on the model hyperparameters specified. A hyperparameter is a parameter whose value is used to control the learning process …

   read more

5. Conditional Colors in Plotly Tables

   Problem

   Generate a data table in Plotly that has the following features:

   1. Alternating cell and line colors for odd/even rows
   2. Unique cell color on first column
   3. For third column onwards, color cells using two different colors based on two levels of upper-bound/lower-bound conditions

   What I did

   Step 1 …

   read more

6. Dataframe manipulation sequence - GroupBy Agg, Melt, Unstack

   Problem

   From a Pandas DataFrame, massage the DataFrame into a format where order Count and Total Amount could be determined for each Vendor and each Vendor-Buyer combination.

   :::python

   >> df = pd.DataFrame(data=
       {'Vendor': ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'C', 'C',
               'D', 'D', 'E', 'E', 'E', 'E', 'E'],      
       'Buyer …

   read more

7. MultiIndex.to_frame()

   Problem

   From a MultiIndex dataframe, determine the total number of elements in the Buyer column for each Vendor.

   What I did

   Let's say we have the following DataFrame:

   :::python

   >> df = pd.DataFrame(data=
       {'Vendor': ['A', 'A', 'B', 'C', 'C', 'C',
               'D', 'D', 'E', 'E', 'F', 'G', 'G'],      
       'Buyer':['BU1', 'BU3 …

   read more

8. MultiIndex.set_levels() in pandas

   Problem

   A user filed an issue on the pandas repo regarding MultiIndex.set_levels - and it turns out the user had some confusion between the set_levels method and the set_names method for MultiIndex due to the documentation. Hence, the MultiIndex.set_levels documentation was marked by the maintainers for improvements to clarify …

   read more