Today I Learned

Sometimes we want to send a docker image to dev or staging environment. That's for testing a small change, for example extra logs for debuging. One could push the image to a registry with some throw away tag, e.g. dev, and then pull it on the other side. But that's unnecessary push to and pull from the registry. Instead, we can save the image to a file and then load it from the file.

# Save image locally
docker save -o my_image.tar my_image:latest
# Load image from a file
docker load -i my_image.tar

If we can ssh into the remote machine then we can easily send the image over.

docker save my_image:latest | gzip | ssh user@remote | docker image load

Now the image my_image:latest is available on the remote machine. Job done easily.

Python has a built in argument parser called argparse. It's the first library most folks use when writing a quick CLI tool. However, it's often quickly abandoned once the app grows in complexity. Unfrotunately, too quickly.

I'm also guilty of too quickly switching to click (nice package) or fire (least favourite evil). I think that's because I have outdated view of what argparse can and cannot do. Just the other day, through accidental laziness I've learned that by default argparse does prefix matching. A flag can be shortened to the shortest unique prefix. For example, if a program has a flag --verbose it can be called with --verb or --verbo or --verbose. This is a very useful feature that I've been missing out on.

Here's a quick example:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--very-big-number', type=float)
parser.add_argument('--verbose', action='store_true')
args = parser.parse_args()

args = parser.parse_args('--very 4242.42 --verb'.split())
print(args)
# Namespace(very_big_number=4242.42, verbose=True)

And if there's an overlap in prefixes then argparse will raise an error:

args = parser.parse_args('--ver'.split())

# usage: test.py [-h] [--very-big-number VERY_BIG_NUMBER] [--verbose]
# test.py: error: ambiguous option: --ver could match --verbose, --very-big-number

Had a use case to checkout a git repository (from Github) via ssh in a Docker image. Turns out that it isn't simple and I'm not entirely sure what's the reason. This entry is a sorrow lesson without a fix. Help is welcome.

In any case, I was expecting this to work but it did not:

FROM ubuntu:latest

RUN apt-get update && apt-get install -y \
    git \
    openssh-client

RUN mkdir -p /root/.ssh && \
    chmod 0700 /root/.ssh && \
    ssh-keyscan github.com > /root/.ssh/known_hosts

RUN ssh-keygen -t rsa -b 4096 -f /root/.ssh/id_rsa -N "" && chmod 600 /root/.ssh/id_rsa

RUN echo "Host github.com\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config

RUN ssh-agent bash -c 'ssh-add /root/.ssh/id_rsa && git@github.com:laszukdawid/tiny-repo.git'

I'm getting the following error:

 > [6/6] RUN ssh-agent bash -c 'ssh-add /root/.ssh/id_rsa && git clone git@github.com:laszukdawid/tiny-repo.git':
0.416 Identity added: /root/.ssh/id_rsa (root@buildkitsandbox)
0.421 Cloning into 'tiny-repo'...
1.173 git@github.com: Permission denied (publickey).
1.175 fatal: Could not read from remote repository.
1.175
1.175 Please make sure you have the correct access rights
1.175 and the repository exists.

Conclusion, for now, is to use https instead of ssh.

To use golang binaries in Python, one can do that by first converting golang programs into shared libraries and then calling them from Python. This can be done by using the cgo package in golang. The following is an example of how to do this:

package main

import "C"

//export Add
func Add(a, b int) int {
    return a + b
}

func main() {}

The above code can be compiled into a shared library by running the following command:

go build -buildmode=c-shared -o libadd.so add.go

The shared library can then be called from Python by using the ctypes package. The following is an example of how to do this:

from ctypes import cdll

lib = cdll.LoadLibrary('./libadd.so')
lib.Add.argtypes = [ctypes.c_int, ctypes.c_int]
lib.Add.restype = ctypes.c_int

print(lib.Add(1, 2))

When working with AWS in GovCloud, it turned out that the boto3 library is not configured to work with GovCloud by default. This is because the GovCloud region is not included in the default list of regions. To fix this, you can add the GovCloud region to the list of regions in the boto3 library. This can be done by adding the following code to your Python script:

import boto3
from botocore.config import Config

config = Config(
    region_name = 'us-gov-west-1',
    signature_version = 'v4',
    retries = {
        'max_attempts': 10,
        'mode': 'standard'
    }
)

client = boto3.client('s3', config=config)