#132 Algorithms as objects

Python Bytes - A podcast by Michael Kennedy and Brian Okken - Mondays

Podcast artwork

Categories:

Technology

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: History of CircuitPython

  • PSF blog, A. Jesse Jiryu Davis
  • Adafruit hired Scott Shawcroft to port MicroPython to their SAMD21 chip they use on many of their boards.
  • CircuitPython is a friendly fork of MicroPython. Same licensing, and they share improvements back and forth.
  • “MicroPython customizes its hardware APIs for each chip family to provide speed and flexibility for hardware experts. Adafruit’s audience, however, is first-time coders. Shawcroft said, “Our goal is to focus on the first five minutes someone has ever coded.” “
  • “Shawcroft aims to remove all roadblocks for beginners to be productive with CircuitPython. As he demonstrated, CircuitPython auto-reloads and runs code when the user saves it; there are two more user experience improvements in the latest release. First, serial output is shown on a connected display, so a program like print("hello world") will have visible output even before the coder learns how to control LEDs or other observable effects.”
  • Related: CircuitPython 4.0.0 released

Michael #2: R Risks Python Swallowing It Whole: TIOBE

  • Is the R programming language in serious trouble? According to the latest update of the TIOBE Index, the answer seems to be “yes.”
  • R has finally tumbled out of the top 20 languages
  • “It seems that there is a consolidation going on in the statistical programming market. Python has become the big winner.”
  • Briefly speculates why is Python (which ranked fourth on this month’s list) winning big in data science? My thought: Python is a full spectrum language with solid numerical support.

Brian#3: The Missing Introduction To Containerization

  • Aymen El Amri
  • Understanding containerization through history
    • chroot jail, 1979, allowed isolation of a root process and it’s children from the rest of the OS, but with no security restrictions.
    • FreeBSD Jail, 2000, more secure, also isolating the file system.
    • Linux VServer, 2001, added “security contextes” and used new OS system-level virtualization. Allows you to run multiple Linux distros on a single VPS.
    • Oracle Solaris Containers, 2004, system resource controls and boundary separation provided by “zone”.
    • OpenVZ, 2005, OS-level virtualization. Used by many hosting companies to isolate and sell VPSs.
    • Google’s CGroups, 2007, a mechanizm to limit and isolate resource usage. Was mainlained into Linux kernel the same year.
    • LXC, Linux Containers, 2008, Similar to OpenVX, but uses CGroups.
    • CloudFoundry’s Warden, 2013, an API to manage environments.
    • Docker, 2013, os-level virtualization
    • Google’s LMCTFY (Let me contain that for you), 2014, an OSS version of Google’s container stack, providing Linux application containers. Most of this tech is being incorporated into libcontainer.
    • “Everything at Google runs on containers. There are more than 2 billion containers running on Google infrastructure every week.”
    • CoreOS’s rkt, 2014, an alternative to Docker.
  • Lots of terms defined
    • VPS, Virtual Machine, System VM, Process VM, …
  • OS Containers vs App Containers

  • Docker is both a Container and a Platform

  • This is halfway through the article, and where I got lost in an example on creating a container sort of from scratch. I think I’ll skip to a Docker tutorial now, but really appreciate the back story and mental model of containers.

Michael #4: Algorithms as objects

  • We usually think of an algorithm as a single function with inputs and outputs.
  • Our algorithms textbooks reinforce this notion.
  • They present very concise descriptions that neatly fit in half of a page.
  • Little details add up until you’re left with a gigantic, monolithic function
  • monolithic function lacks readability
  • the function also lacks maintainability
  • Nobody wants to touch this code because it’s such a pain to get any context
  • Complex code requires abstractions
  • How to tell if your algorithm is an object
  • Code smell #1. It’s too long or too deeply nested
  • Code smell #2. Banner comments
  • Code smell #3. Helper functions as nested closures, but it’s still too long
  • Code smell #4. There are actual helper functions, but they shouldn’t be called by anyone else
  • Code smell #5. You’re passing state between your helper functions
  • Write your algorithm as an object
  • Refactoring a monolithic algorithm into a class improves readability, which is is our #1 goal.
  • Lots of concrete examples in the article

Brian #5: pico-pytest

  • Oliver Bestwalter
  • Super tiny implementation of pytest core. 25 lines
  • My original hand crafted test framework was way more code than that, and not as readable.
  • This is good to look at to understand the heart of what test frameworks do
    • find test code
    • run it
    • mark any exceptions as failures
  • Of course, the bells and whistles added in the full implementation are super important, but this is the heart of what is happening.

Michael #6: An Introduction to Cython, the Secret Python Extension with Superpowers

  • Cython is one of the best kept secrets of Python.
  • It extends Python in a direction that addresses many of the shortcomings of the language and the platform, such as execution speed, GIL-free concurrency, absence of type checking and not creating an executable.
  • Number of widely used packages that are written in it, such as spaCy, uvloop, and significant parts of scikit-learn, Numpy and Pandas.
  • Cython makes use of the architectural organization of Python by translating (or 'transpiling', as it is now called) a Python file into the C equivalent of what the Python runtime would be doing, and compiling this into machine code.
  • Can sometimes avoid Python types altogether (e.g. sqrt function)
  • C arrays versus lists: Python collection types (list, dict, tuple and set) can be used as a type in cdef functions. The problem with the list structure, however, is that it leads to Python runtime interaction, and is accordingly slow
  • Nice article for getting started and motivation. But I didn’t see Python type annotations in play (they are now supported)

Extras

Brian:

Michael:

Jokes

  • What do you call eight hobbits? A hobbyte.
  • Two bytes meet. The first byte asks, 'Are you ill?' The second byte replies, 'No, just feeling a bit off.’
  • OR: What is Benoit B. Mandelbrot's middle name? Benoit B. Mandelbrot.