Today I want to share with you something I’ve been doing in the last couple of days. The lock down measures for COVID-19 and some free time brought me to tinker a bit with my Raspberry Pi4.

The project Rosetta@home allows people to give compute power to scientists for designing new proteins. The compute power could come from any device: a laptop, a desktop, a server and even a portable device.

Recently I bumped into the news of running Rosetta@home on aarch64 (arm64) hardware and, in my case, on my new and shiny Raspebrry Pi4.

In theory what you need is:

• An account on: https://boinc.bakerlab.org
• A Raspberry Pi4
• A 64-bit OS
• The BOINC client

  $ sudo apt-get update
  $ sudo apt-get install boinc-client boinctui
  

• The client configure with your email/password

A good write-up could be found here

While I was setting it up I found few “issues”. Hopefully they could save you some time.

Rosetta doesn’t work on Raspbian
If you own a Raspberry Pi4, I’m sure you’ll have a Raspbian based system. It runs without any problem but unfortunately it is not suitable for running Rosetta@home. Although the hardware supports 64-bit, Raspbian is a 32-bit OS and this means it cannot run 64-bit applications.
Solution: install a 64-bit distribution.
Ubuntu officially supports Raspberry Pi4. Download the 64-bit version from here or follow this tutorial from Ubuntu I had issue with WiFi module hence reverted to wired connection!

The SD card is a bit slow.
Although SD cards nowadays are fast, they are slower than HDD/SDD and when you run high I/O computations on them, they could fail fairly quick.
Solution: move the root file system out from the SD card to an external hard disk.
I had a 1TB portable one and it works like a charm. It’s not that straightforward to move the root file system to an external disk but it’s a doable. What you need is a good guide

The Pi4 is slow.
Those are the specs. As you can see it has 4 CPUs running at 1.5GHz. Not bad for a small board.
Solution: overclock the CPU.
Just follow this tutorial. I did try to overclock it to 2GHz but it kept power cycling (likely due to power consumption) so I stuck with 1.75GHz.
If you overclock it, I strongly suggest something to cool down the CPU. I have a Fan SHIM: a bit noisy but it does the job.

High memory usage
Even I have the 4GB version, I found the Raspberry Pi4 to be very unstable while doing computations. Memory is somehow limited and when processes use a large amount of it, the OOM killer could be invoked killing processes that make the system unstable.
Solution: create a swapfile
Like any other Linux distribution create swap space to use part of the hard disk as “extention” of the memory. I created a 4GB swap file and it seems working OK.

“PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python or its processes or environment. The PEP should provide a concise technical specification of the feature and a rationale for the feature” (from PEP 1)

There are three kinds of PEP:

• A Standards Track PEP describes a new feature or implementation for Python
• An Informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature
• A Process PEP describes a process surrounding Python, or proposes a change to (or an event in) a process. Process PEPs are like Standards Track PEPs but apply to areas other than the Python language itself

All PEPs are stored in a github repository and the list is in the PEP 0

Now, the main question is: “What are the most important PEPs that a Python developer should be aware of?” I went through the long list and I chose the ones that I think are important to know.

The must know PEPs without witch you cannot be considered a Python programmer are:

• PEP 20 – The Zen of Python: that’s the first PEP you need to read and apply throughout your Python development
• PEP 8 – Style Guide for Python Code: this PEP defines the coding convention when writing Python. I expect everyone writing PEP8 compliant code
• PEP 257 – Docstring Conventions: conventions for Docstring in Python
• PEP 287 – reStructuredText Docstring Format: when plain text is not enough, reStructuredText is the answer for Docstring

The following PEPs don’t fall into any category but it’s good to know their existence anyway:

• PEP 7 – Style Guide for C Code: if you need to contribute to C implementation of Python, that’s the PEP for you
• PEP 404 – Python 2.8 Un-release Schedule: release not found! There will be never a 2.8 Python release. The last Python 2 release is 2.7. It’s time to move to Python 3!
• PEP 440 – Version Identification and Dependency Specification: this PEP describes a scheme for identifying versions of Python software distributions, and declaring dependencies on particular versions

The following PEPs instead will be split between Python versions (2 and 3) and they just represent an attempt to summarise the most relevant ones in order to pick peculiarities of the language itself. The version of python affected and some personal extra comments will be side noted.

Python 2

• PEP 201 – Lockstep Iteration (zip function) (2.0) - it’s very handy
• PEP 202 – List Comprehensions (2.0) - one of my favourites
• PEP 221 – Import As (2.0)
• PEP 234 – Iterators (2.1) - trust me, you will use them
• PEP 236 – Back to the __future__ (2.1) - do you want to use print() (from Python 3.X) in Python 2.X? That’s the module for you
• PEP 255 – Simple Generators (yield) (2.2) - once you know it, you’ll love it
• PEP 279 – The enumerate() built-in function (2.3) - sometime it is very useful
• PEP 282 – A Logging System (2.3) - please, don’t debug with print()
• PEP 285 – Adding a bool type (2.3) - I don’t understand how you live without it!
• PEP 289 – Generator Expressions (2.4) - list comprehensions with generators
• PEP 308 – Conditional Expressions (2.4) - Python “ternary” expression: X if C else Y
• PEP 318 – Decorators for Functions and Methods (2.4) - @decorators are methods which wrap functions and methods
• PEP 322 – Reverse Iteration (2.4)
• PEP 324 – subprocess - New process module (2.4)
• PEP 327 – Decimal Data Type (2.4) - floatintg point are just too inexact. Period.
• PEP 341 – Unifying try-except and try-finally - Do you know that try constructs allow an else statement?
• PEP 342 – Coroutines via Enhanced Generators (2.5) - still need to get those!
• PEP 343 – The “with” Statement (2.5) - really? Don’t you know it? Shame on you!
• PEP 367 – New Super (2.6) - that’s important, read it!

Python 2 and 3

• PEP 274 – Dict Comprehensions (originally 2.3, then 2.7 and 3.0) - the beauty of list comprehensions applied to Dict
• PEP 372 – Adding an ordered dictionary to collections (2.7, 3.1) - because order matters
• PEP 389 – argparse - New Command Line Parsing Module (2.7, 3.2) - please, stop using optparse right now!

Python 3

• PEP 380 – Syntax for Delegating to a Subgenerator (yield from) (3.3) - a generator to delegate part of its operations to another generator
• PEP 405 – Python Virtual Environments (3.3) - I’m really glad to see a tighter integration with virtual environments
• PEP 417 – Including mock in the Standard Library (3.3) - it’s one of the first modules I pip install when using Python 2.7
• PEP 435 – Adding an Enum type to the Python standard library (3.4)
• PEP 450 – Adding A Statistics Module To The Standard Library (3.4) - never used so far, but I recognise its importance
• PEP 483 - The Theory of Type Hints (3.5) - that’s just a theory for the next PEP
• PEP 484 - Type Hints (3.5) - they are used to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information. Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention
• PEP 485 – A Function for testing approximate equality (3.5) - is_close enough :)
• PEP 492 – Coroutines with async and await syntax (3.5) - coroutines start being a proper concept in Python
• PEP 525 – Asynchronous Generators (3.6) - generators made asynchronous
• PEP 526 – Syntax for Variable Annotations (3.6) - it’s like PEP 484 but for variables
• PEP 530 – Asynchronous Comprehensions (3.6) - things are getting complicated eh… asynchronous versions of list, set, dict comprehensions and generator expressions
• PEP 3000 – Python 3000 (3000) - where 3v3rything begins!
• PEP 3101 – Advanced String Formatting (3.0) - stop using % for string formatting and embrace .format() method (included in Python 2.6 as well)
• PEP 3105 – Make print a function (3.0) - I told you not to use print for debugging! Now all your Python 2.X needs a huge refactoring for working in Python 3.X. Joking apart, that’s the most common error when you try to run Python 2.X code using Python 3.X intepreter.
• PEP 3107 – Function Annotations (3.0)
• PEP 3109 – Raising Exceptions in Python 3000 (3.0) - yep, in Python 3.X, raise changes a bit. It’s worth having a look
• PEP 3110 – Catching Exceptions in Python 3000 (3.0) - see above
• PEP 3115 – Metaclasses in Python 3000 (3.0) - well, if you have ever used __metaclass__, Python 3.X don’t
• PEP 3119 – Introducing Abstract Base Classes (3000) - abc, no I haven’t forgotten the alphabet :)
• PEP 3120 – Using UTF-8 as the default source encoding (3.0) - this PEP removes a lot of headeaches we have in Python 2.X
• PEP 3129 – Class Decorators (3.0) - like PEP 318 but for classes
• PEP 3135 – New Super (3.0) - it’s PEP 367 for Python 3.X
• PEP 3148 – futures - execute computations asynchronously (3.2) - The concurrent.futures module provides a high-level interface for asynchronously executing callables
• PEP 3156 – Asynchronous IO Support Rebooted: the “asyncio” Module (3.3) - this is it! the asyncio module

Have I missed anything? What’s your favourite PEP?

With Github pages you can set up a website for every project you have on Github or you can set up a “personal” website (linked to your username.github.io repository on Github).

The underlying technology is Jekyll (yes, written in Ruby… :-/). You need to install it on your computer is you are migrating an existent website to Jekyll or if you want to customise your website even more. The reasons I move to Github Pages are summed in the following video

The old website was composed from:

• digitalocean.com server, which I strongly suggest (grab my referral!)
• Wordpress installation (theme and software updates)
• Mysql installation (backups, data restore, schema upgrades, etc…)
• scripting around maintenance

The overhead for a simple website was simply too much: software updates and upgrades, schema migrations, backups, data restores, dying processes, dying processes and many other tasks which were time consuming.

The new website is composed from:

• git repository (yes, that’s right!) :)

No databases, no schemas, no servers, no passwords, no hacks, no software updates, no dying processes, no downtime (well there might be, but Github will fix it for me) and backups are already included!

So if you already use Github, wait no more! Set up your website now!

If you want to migrate your existent website to Jekyll… Well, good luck! :) Joking apart, the process might not be so straightforward as a lot of things need to be ported/configured:

• posts and pages: Jekyll is blog aware so you can export to it from the majority of blog softwares (check it here). Although my blog was on Wordpress this took me plenty of time as now always the export works as expected. I’ve used both scripts and manual work to do a proper migration
• comments: as there is no database you need to use an external service. I’ve chosen DISQUS
• URLs: I’ve mapped as much as I could the old URLs with the new ones
• theme: I went for Minimal Mistakes as the documentation is just excellent and I love the clean design!
• custom code: depends how many plug-ins you used in your previous platform, you might end up in writing plenty of custom code to keep the same functionalities

Well, if you have any question I bet you can find my contact or comment this post

Blogging with VIM and publishing with git push is just priceless! :)

micro:bit is a board that BBC will distribute to every year 7 student in the UK (early 2016, roughly 1 million units) and it will be shipped with micropython on it (more info). Hardware specs can be found here: it has a Cortex-M0, compass, accelerometer, 5x5 leds, bluetooth, uart, i2c, two buttons and other stuff you can use.

In a such tiny little board you can run python… well micropython, a fast implementation of Python 3 programming language that is optimised to run on a microcontroller.

So, what did we do? Once plugged the micro:bit to the USB, we can run python code through microrepl, a script that lets you run commands directly onto a connected micro:bit device. Alternatively you can use the browser based micro:bit editor, and start programming directly in your browser. Very neat!

My group (Gareth and myself), after exploring, experimenting, and testing the APIs, decided to do something with the 5x5 leds available. This is the code we have written literllay in few minutes:

import microbit     


# It returns the brightness (0-9) depending on the position of the led  
def brightness(x, y, t):  
    return (x + y + t) % 10


# Iterate the whole led matrix and set the pixel with the right brightness  
def get_image(t):  
    for x in range(0, 5):  
        for y in range(0, 5):  
            microbit.display.set_pixel(x,y, brightness(x,y,t))                           


t = 0  # time  
a = 1  # direction flag


# Start looping  
while True:  
    get_image(t)  # Set all the leds  
    t += a  # Incrementing t  
    microbit.sleep(100)  # Let's wait some time  
    if microbit.button_a.is_pressed():  #Check if the user presses the button  
        a *= -1  # Revert the direction  

This is what it does (in real time the effect is much better):

And this is the slow motion:

When the user presses the button, it changes direction. It has been very good fun and I’m sure kids will love it: it is straightforward to use it, pretty intuitive and microrepl is very easy to use (it has autocompletion!)

In /etc/bash_bashrc_Apple_Terminal (which is called by /etc/bashrc), the following piece of code can cause some issues when you have set HISTFILESIZE and HISTSIZE to a very high values:

# Save new history commands.  
if [ $BASH_SESSION_HISTORY -eq 1 ]; then  
    history -a  
    cat "$BASH_SESSION_HISTFILE_NEW" >> "$BASH_SHARED_HISTFILE"  
    cat "$BASH_SESSION_HISTFILE_NEW" >> "$BASH_SESSION_HISTFILE"  

    # Empty this session's history file to keep track of  
    # which commands have already been copied.  
    : >| "$BASH_SESSION_HISTFILE_NEW"

    # Read/write the files via the history command so they  
    # are truncated as appropriate.  
    history -r "$BASH_SHARED_HISTFILE"  
    history -w "$BASH_SHARED_HISTFILE"  
    history -r "$BASH_SESSION_HISTFILE"  
    history -w "$BASH_SESSION_HISTFILE"  
fi  

I had the HIST* variables set as:

HISTFILESIZE=1000000000  
HISTSIZE=1000000

and this is what happened when I logout from the shell.

leonard:~ diegor$ logout  
+ bash_update_session_state  
+ bash_save_session_state  
+ '[' 1 -eq 1 ']'  
+ '[' -n /Users/diegor/.bash_sessions/97360883-85FB-4463-A9DF-1246FCD05977.session ']'  
+ echo -n Saving session...  
Saving session...++ date  
+ echo echo Restored session: Mon Jul 13 08:23:07 BST 2015  
+ declare -F bash_session_save_state  
+ '[' 1 -eq 1 ']'  
+ history -a  
+ cat /Users/diegor/.bash_sessions/97360883-85FB-4463-A9DF-1246FCD05977.historynew  
+ cat /Users/diegor/.bash_sessions/97360883-85FB-4463-A9DF-1246FCD05977.historynew  
+ :  
+ history -r /Users/diegor/.bash_history  

Looking at the help of history, it is stated:

The `-w' option writes out the current history to the history file;  
`-r' means to read the file and append the contents to the history list instead.

which means: if you HIST* size are very high, the size is going to grow a lot, with some implications (see three bullet points below)

At this point you observe the following things:

• the terminal is stuck on Saving session...
• CPU usage goes high (you have to kill the bash process from another shell or System Monitor)
• the size of ~/.bash_session is increasing overtime: every time you logout it increases by the size of the history file (in my case was 20MB)

My suggestion is NOT to set those variables and let OSX deal with them.

IHTH

Nothing, nada, tabula rasa.

I couldn’t boot to Linux unless I went to the kind of BIOS menu and selected the Linux partition. After spending few days after this, I bumped into this comment in a stackoverflow post. It states:

“Yes! Renaming worked for me! I mounted the efi partition and copied the files ../EFI/ubuntu/grubx64.efi to ../EFI/Boot/ and to /EFI/Microsoft/Boot and changed the name of the file as follows. In the ../EFI/Boot directory I overwrote the bootx64.efi with the file and in the ../EFI/Microsoft/Boot I overwrote the bootmgfw.efi with the file. That was all I think. Thank you very much! - user296880 Jun 23 ‘14 at 20:28”

Well user296880, it did work for me too! Renaming file under the hood is really the last resort. A new upgrade might wipe everything and have to start again but I accept the risk.

Searching a bit better I also bumped into this other post which explains exactly my situation.

Said that, really good luck with that.

This method will rewrite the history of your repository. If someone pulled/fetched from the repo between the commit with the wrong message and when you correct it, he/she could have some issues. It would be useful to communicate to anyone involved about your action.

$ setxkbmap

That’s it.

$ echo $STY

If you are in a screen session, you’ll see something similar to

$ echo $STY  
5413.ttys001.leonard

If you aren’t, you’ll see nothing.

$ echo $STY

$