Blog

As we get older, our thoughts solidify a bit. We are unintentionally drawn into more experiments, we get more data, and we learn more theories that help us to make sense of the world. If we're lucky, these experiments are diverse enough to sample from "real-life" distribution, rather than repeating the same situation over and over again. If we're super lucky, we also get to learn about the Central Limit theorem.

Resolutions are a double-edged sword. On one hand, they give us hope and motivation to do something new. On the other hand, they can be difficult to achieve because they are typically big goals. It's hard to change oneself into a new mold without a driving motivation and stimuli. We're used to our day-to-day routines, and if we don't pay attention or are under stress, we'll default to them.

Unsupervised learning is a powerful technique in machine learning, particularly when dealing with datasets where labeled data is unavailable. It allows the model to identify patterns, structures, and relationships in the input data without any human supervision or guidance. This can be particularly useful for tasks such as categorizing images, identifying anomalies in time series data, or detecting outliers in graphs.

ML infrastructure plays a crucial role in the success of a machine learning project. Building and maintaining ML infrastructure can be complex and time-consuming, especially when started from scratch for each project. Obviously, horses for courses; some projects are one-offs and optimising might derail your enthusiasm, and some projects are so large that everything becomes an exception. Most projects, however, are (un)fortunately going to be very similar from a high-level perspective. Each will need some data storage, computation units and means to deploy models. Here are a few short tips.

In less than 12 hours after publishing this note I'll be talking to my new colleagues at my new job. About a year ago was the last time that I talked with colleagues about job related stuff. Rudimentary maths - I haven't worked for anyone for over a year. Below are some thoughts and feelings regarding this gap.

It's been over a year since I left a stable and well paying job, and focused more on myself. A year feels like a nice unit of time to summarize over but I didn't feel the need to say something. Until now. Maybe it’s worth sharing what I’ve been sharing with everyone who wanted to listen.

There are a few things that I'd like to announce and preserve for accountability. Unfortunately, Agents Bar News page isn't ready yet so my blog will have to do. Hopefully the following newsletters are "more professional".

A downbeat this one is.

It officially started a few months ago. The execution plan was formed over a year ago. The expected long term result is with me for circa 20 years. Though the rest might sound like I'm giving up, I certainly am not. Just following internet customs of shouting out one's intentions, thus avoiding prolonged gossips by squeezing them into a burst. Ay, clever, ain't it?

One of those notes written under a sudden and noticeable motivation spike. I'll attribute it to coffee but it's obviously not the only factor. I had coffee many times in the last week but this feeling came maybe once or twice. Something needs to be attributed and as the author I need to give you an anchor, otherwise you'll go on a wild hunt.

What am I talking about? Yes. I'm with you on this question. (No, this post is not written by GPT-X.)

It’s a huge pleasure to announce, yet it is rather scary. As in title – Agents Bar is now in public beta. Go ahead, sign up, sign in and try it. I’d love to hear what you think.

TensorBoard seems to be a de facto standard tool for displaying your ML experimental data. There are some alternatives, but likely due to "being first" and "by google" stamps the TensorBoard (TB) is still the most common one. Many of these alternatives are far superior, especially if you're using something else than Tensorflow, but TB is still the first go to.

Usually around this time I spend more days actively thinking and writing. As a boring person I write predominately about ideas for the future, conclusions from the past, experience to repeat, how others should conduct their lives... The boring stuff. Since this is rather cyclical my guess is that it's mainly because nights are coming early and they bring quietness, bring focus. Coming of the holidays and the new year is also a trigger for reminiscence and planning.

Whenever I'm running PyTorch with enabled CUDA and laptop decides to turn off/put to sleep the GPU, executed script will fail with some CUDA allocation error. Not cool, laptop, super not cool. Restarting the script, or starting any other program that uses CUDA, might not be possible since the memory wasn't properly released. When trying to run PyTorch you might see the same error as in the title, i.e.

RuntimeError: cuda runtime error (999) : unknown error at /pytorch/aten/src/THC/THCGeneral.cpp:47

I saw a few people referring to S. Zayd Enam's article as an explanation of why machine learning (ML) is hard. It is a good read with some visuals so I'd recommend reading it. However, I don't entirely agree with author's arguments and so I thought about highlighting a few nuances.

It's been over a week since I left Amazon where I've worked for over three years. It seems like a right time to write out some thoughts on the journey.

tl;dr: AiTraineree is a new Deep Reinforcment Learning lib based on PyTorch.

tl;dr: A GitHub gist with commands walk-through is available here.

I've been evolving my productivity process for a long time and there are a few aspects that really work for me. One of them is doing official starting and ending ceremonies. By official I mean a session where you pretend to report findings to your boss although they and everyone else are (suspiciously) quite. Often that report means going through the project template and discussing all notes. Then, ceremonially move the project file/note to a different directory with all other completed (not necessarily successfully) projects and writing out learnings.

tl;dr: PyEMD documentation has a section on speeding up tweaks.

Presenting to you one of the side projects: the Personal Progress.

Yep, it's one of those titles where unless you know what it's about you probably don't know what it's about. If you feel with these abbreviations, go ahead and scroll down a bit. Otherwise, let me go ahead and expand here a bit: Amazon Web Services' Cloud Development Kit for Elastic Container Service Fargate. That's probably not clearer so on to the descriptions we go.

In the software world, there's a saying "if you want users to get excited about a new release, change buttons' colours." (I'm saying this.) Go ahead, look around; looks different, right?

Despite my strong opinion against the AWS Glue with its unfriendly documentation and strange approach to anything... I ended up using it as the main framework for an AWS native ETL (Extract, Transform, Load) service. The whole journey felt like trying to make divorced parents get back together. They're working together but the process felt artificial and not sure whether they're meant for each other. The success was due to finding out Glue's some dirty secrets.

Most of my projects now are separated from the beginning into logical entities and put into docker. This makes the deployment and replacement much easier to manage but the development many rituals comands.

I've been postponing describing my experience with AWS Glue for a couple of months now. It's one of those things that I really wanted to get out of myself but it hurts to even speak up. Let's end this pain, let's end this now. Ehkem... AWS Glue sucks.

Let me share with you the thing I found on the internet. That thing is called Exist.io. They aren't paying me for this ad but I'm still going to do it as it reignited my push for the quantified self.

There has been a surprising shift in my approach to writing. Any external request to express or clarify something in a written for would evoke a passionate hatred. I could feel the amount of time that is going to be wasted for no good reason what-so-ever. What others might complete in 5 min I would need to spend more than an hour. The reason was that many people considered my expression style as "weird" or "unusual", and would often ask for clarifications in the way they wanted just to make sure that we are on the same page. Talking is easier because you quickly clarify these concerns and carry on. Writing, however, is a double pain. I'd spend hours on rewiring and polishing a couple of sentences trying to clarify what might be potentially unclear and, then, I'd spend a couple on replying to feedback which typically meant "unclear." It's a frustration spiral.

Having more time I started retrospect on my previous projects and attempt to extract characteristics that lead either to success, or failure. As one might expect, these characteristic change in time. Currently, I’m doing better off dividing work into chunks and deal one at the time but a while back it was easier to just get into working and the immediate task would pop out. Going forward my habits and motivation will definitely change again so it’s difficult to impose a strict ‘successful project’ template, but, we aren’t setting things in stone and if needed we’ll change them.

Working for a tech behemoth corp is significantly different than what I expected joining. There is plenty of corpo bullshit, "vapor-ware" and ass-lickers that simply want to game out the system to climb the ladder. But, obviously, that's not all it has. Can't tell how is everywhere, but where I am, I must admit that I'm surrounded mainly by competent people and well-matured ideas.

There was a significant update on my title. Since the end of November, I am officially a PhD. The relief is immense. Obviously, life goes on and nothing has significantly changed on the outside but I can see that my approach to things lighten up and the approach of "Yes can do" returned. I'm open to new projects and ideas.

In Ubuntu 18.04, typing in Nautilus file manager makes a deep search through it's all subdirectories. This is potentially a powerful and convenient tool but, unfortunately, not for me as it doesn't fit into my workflow. I'm less of the GUI fan and only use a graphical file manager when I need to do some quick drag and drops on file bulks. Typically, if I'm using Nautilus, I'm already in the directory where I need to be and just want to quickly jump to a file/dir by the name. Shortly: yes, bring back the ye olde stuff.

The last post about AWS EMR and S3 has resulted in a few people messaging me directly. To ease others let me add something about how I approach a specific problem.

One of the recent projects I've worked on involved processing billions of row stored in AWS S3 in terabyte size data. That was the biggest I've worked so far and, even though I don't like the term, it broke through the big data barrier. Just handling the data with popular toolkits, such as scikit-learn or MXNet, created so many problems that it was easier to create our own solutions. But the biggest surprises came from places least expected.

Although learning and book knowledge are the best, my personal relationship with reading activity is not the friendliest. Being focused on the text is a huge struggle and I often need to re-read sentences to actually read it. That's why sometimes I use text-to-speech (TTS) software or service.

Quick ad hoc command:

:%!python -m json.tool

com! FormatJSON %!python -m json.tool

Chances are you heard that multiprocessing in Python is hard. That it takes time and, actually, don't even try because there's something like global interpreter lock (GIL), so it isn't even true parallel execution. Well, GIL is true, but the rest is a lie. Multiprocessing in Python is rather easy.

CEEMDAN is available in Python through PyEMD.

There are as many empirical mode decomposition (EMD) variations as many teams are working on it. Everyone notices that in general EMD is very helpful method, yet, there's room for improvement. Few years back I have stopped doing modifications myself in exchange for working on mathematically sound model of coupled oscillator. Nevertheless, since I spent quite a lot of time on EMDs and have enjoy playing with it, from time to time something will catch my eye. This is what happened with Complete Ensemble EMD with Adaptive Noise (CEEMDAN).

My "writing" work currently goes somewhere else and have little motivation to write anything here. But, there's something that only internet can help, whether that's through actual help or simply transferring my annoyance.

Amazon with it's Amazon Web Service (AWS) is pretty cool. It gives you access to remote machine which you don't have to maintain. Actually you don't have to do anything other than use it. All machines come in different flavours, but what tastes better than free? Granted that it's extremely limited, but surely we can squeeze something out of it. Right?

One of the packages I intend long term maintain and support is Python implementation of Empirical Mode Decomposition (EMD) called PyEMD. I will skip introduction of the method as it has been explained in few other posts [1, 2, 3, ...]. This blog entry is more about announcement of new feature which also means new version.

I'm becoming increasingly convinced that using GitHub to share projects, even in work-in-progress (WIP) state is quite beneficial. For one, someone can quickly point out that such project exists or suggest a tool which would simplify some operations. What's more, with tools like Travis-CI or CodeCov it's easy to follow whether the project is actually buildable or whether updates broke functionally of other features. Real convenience! Although, there is some price to pay for these goods.

tl;dr: Project on github: https://github.com/laszukdawid/pystan-kuramoto

Only recently I've started using VIM as my default editor and... I'm amazed.

I'm not advocating here for its superiority over any other text editor. You might be into design or simply wish for a graphical interface. I'm into not-using-mouse too much, so it actually fits quite well my purposes. Having said that, it took me a while to discover it with more commands than saving and exiting.

Those who know me, know that I'm not a fan of using mouse. Whenever possible I try to avoid using it. Although, after years of experience, I am fluent in keyboarding there are still some tasks that I need mouse. Well, until quite recent.

The number one of annoying tasks is copying things from terminal to clipboard. Depending what exactly I needed to do with it I'd either select something with mouse and then copy, or stream output to file and then select it within editor. Some people found it weird, but yes, often copying through editor is much faster.

Code for Kuramoto in Python is available here or from code subpage. Explanation on how to use it is on the bottom of this post.

It came to my attention that my PSO for Python is actually quite popular. Only after few people contacted me I've noticed that the public version was not the same that I've been using. It wasn't bad, but definitely not as good. Thus, obviously, I've updated the version and cleaned it a bit.

Recent challenge I had was to convert two given numpy arrays such it'll be possible to have them in common type without losing information. One can follow this link for stackoverflow entry.

Sometimes when we ask for random we don't actually mean random by just random. Yes, pseudo-random.

Consider unitary distribution with ranges 0 and 1. Say you want to draw 5 samples. Selecting them at random would mean that we might end up with set of {0, 0.1, 0.02, 0.09, 0.01} or {0.11, 0.99, 0.09, 0.91, 0.01}. Yes, these values don't seem very random, but that's the thing about randomness, that it randomly can seem to not be random.

Only recently I have started to use Python 3. It's been out for good 8+ years and all these excuses about incompatibility with some packages were just lazy. Most packages I use are already ported and if I ever find that something is incompatible... well, I'll think then. But for now let me pat myself on the back for this great leap, because:

A while ago I've started to taste a bit how it feels to work in industry and it feels quite nice. Maybe that's the specificity of field projected onto the industry, or being tired of how academia works, but I'm enjoying extremely learning all the details about Computer Science, programming and newest technologies.

I'm an avid podcast listener. Whenever there's something that only requires sight or not much focus I'll try to do it with my headphones on. Great thing about podcasts is that they they are more up-to-date than audiobooks and have reasonably short lengths, so there's always a fit.

tl;dr: Read subject and click on link below.

Some of you, i.e. those lucky ones with connection to the outside World, are probably aware about Machine Learning community trying to aggressively change our lives for better. Regardless whether we like it or not, they're doing it. Some do this for money, others for fame, and those wicked ones just for fun.

This blog needs some lightening up and what a better way to do that than post some graphs? Exactly! I like to collect data on various things and then make a  graph of them. Things are much better when presented with axis and some number around it.

tl;dr: I'm going less quality, more quantity.

I like to write. I write lots, but usually don't publish it. Whenever I think about something and write it down, I then rethink what I thought and want to rewrite what I wrote. It takes me ages to write something very precise and something that I wouldn't be immediately ashamed of. This blog was meant to be the place for that content, for things I wouldn't quickly regret. And, although, I'm rather OK with the content, I'm deeply disappointed with the frequency. Thus: change.

Another of my papers [1] (see About) has been published recently. Unfortunately, I cannot present it here in a full scope. Below I'm presenting an abstract from the paper and invite interested people in contacting me. This topic is really interesting and there is plenty to be done.

For the past few years I've been teaching few modules for Master students, such as programming or signal processing. In most of them, one of the main and important tasks is to write a data buffer. Usual requirement is to be able to return last N samples or T seconds. However, conceptually most of students understands, implementing this seems to be quite difficult. But it is not. Unfortunately their first line of action, rather than think themselves, is to search for the solution on the Internet. There are many attempts, better or worse, but since they are looking for it, I could also provide with my own solution. Hopefully it helps others as well.

For a while now, I've been involved in Kuramoto models and adjusting them to data. Recently I've stumbled upon using Bayesian inference on predicting parameters in time-evolving dynamics. Team from Physics Department at University of Lancaster has produced many papers, but to provide with some reference it might be worth to look at [1].

[Updated version available. See newest post.]

tl;dr: For Python PSO code head to codes subpage.