HomeHero - next iteration
Exciting news everyone! Reveal of the next iteration on the HomeHero - my household management - app. It's no longer ugly proof of concept. Some people are actually using it. And it's getting better!
Exciting news everyone! Reveal of the next iteration on the HomeHero - my household management - app. It's no longer ugly proof of concept. Some people are actually using it. And it's getting better!
A set of principles that should align for successful project.
HomeHero - the app
Coming soon - HomeHero
As we get older, our thoughts solidify a bit. We are unintentionally drawn into more experiments, we get more data, and we learn more theories that help us to make sense of the world. If we're lucky, these experiments are diverse enough to sample from "real-life" distribution, rather than repeating the same situation over and over again. If we're super lucky, we also get to learn about the Central Limit theorem.
Read MoreResolutions are a double-edged sword. On one hand, they give us hope and motivation to do something new. On the other hand, they can be difficult to achieve because they are typically big goals. It's hard to change oneself into a new mold without a driving motivation and stimuli. We're used to our day-to-day routines, and if we don't pay attention or are under stress, we'll default to them.
Read MoreUnsupervised learning is a powerful technique in machine learning, particularly when dealing with datasets where labeled data is unavailable. It allows the model to identify patterns, structures, and relationships in the input data without any human supervision or guidance. This can be particularly useful for tasks such as categorizing images, identifying anomalies in time series data, or detecting outliers in graphs.
Read MoreML infrastructure plays a crucial role in the success of a machine learning project. Building and maintaining ML infrastructure can be complex and time-consuming, especially when started from scratch for each project. Obviously, horses for courses; some projects are one-offs and optimising might derail your enthusiasm, and some projects are so large that everything becomes an exception. Most projects, however, are (un)fortunately going to be very similar from a high-level perspective. Each will need some data storage, computation units and means to deploy models. Here are a few short tips.
Read MoreIn less than 12 hours after publishing this note I'll be talking to my new colleagues at my new job. About a year ago was the last time that I talked with colleagues about job related stuff. Rudimentary maths - I haven't worked for anyone for over a year. Below are some thoughts and feelings regarding this gap.
Read MoreIt's been over a year since I left a stable and well paying job, and focused more on myself. A year feels like a nice unit of time to summarize over but I didn't feel the need to say something. Until now. Maybe it’s worth sharing what I’ve been sharing with everyone who wanted to listen.
Read MoreThere are a few things that I'd like to announce and preserve for accountability. Unfortunately, Agents Bar News page isn't ready yet so my blog will have to do. Hopefully the following newsletters are "more professional".
Read MoreA downbeat this one is.
It officially started a few months ago. The execution plan was formed over a year ago. The expected long term result is with me for circa 20 years. Though the rest might sound like I'm giving up, I certainly am not. Just following internet customs of shouting out one's intentions, thus avoiding prolonged gossips by squeezing them into a burst. Ay, clever, ain't it?
Read MoreOne of those notes written under a sudden and noticeable motivation spike. I'll attribute it to coffee but it's obviously not the only factor. I had coffee many times in the last week but this feeling came maybe once or twice. Something needs to be attributed and as the author I need to give you an anchor, otherwise you'll go on a wild hunt.
What am I talking about? Yes. I'm with you on this question. (No, this post is not written by GPT-X.)
Read MoreIt’s a huge pleasure to announce, yet it is rather scary. As in title – Agents Bar is now in public beta. Go ahead, sign up, sign in and try it. I’d love to hear what you think.
Read Moretl;dr: I'm looking for co-founders to help push the Agents Bar to glory. Everyone deserves an easy access to deep reinforcement learning (DRL) agents in their applications.
Read MoreTensorBoard seems to be a de facto standard tool for displaying your ML experimental data. There are some alternatives, but likely due to "being first" and "by google" stamps the TensorBoard (TB) is still the most common one. Many of these alternatives are far superior, especially if you're using something else than Tensorflow, but TB is still the first go to.
Read MoreUsually around this time I spend more days actively thinking and writing. As a boring person I write predominately about ideas for the future, conclusions from the past, experience to repeat, how others should conduct their lives... The boring stuff. Since this is rather cyclical my guess is that it's mainly because nights are coming early and they bring quietness, bring focus. Coming of the holidays and the new year is also a trigger for reminiscence and planning.
Read MoreWhenever I'm running PyTorch with enabled CUDA and laptop decides to turn off/put to sleep the GPU, executed script will fail with some CUDA allocation error. Not cool, laptop, super not cool. Restarting the script, or starting any other program that uses CUDA, might not be possible since the memory wasn't properly released. When trying to run PyTorch you might see the same error as in the title, i.e.
Read MoreRuntimeError: cuda runtime error (999) : unknown error at /pytorch/aten/src/THC/THCGeneral.cpp:47
I saw a few people referring to S. Zayd Enam's article as an explanation of why machine learning (ML) is hard. It is a good read with some visuals so I'd recommend reading it. However, I don't entirely agree with author's arguments and so I thought about highlighting a few nuances.
Read MoreIt's been over a week since I left Amazon where I've worked for over three years. It seems like a right time to write out some thoughts on the journey.
Read Moretl;dr: AiTraineree is a new Deep Reinforcment Learning lib based on PyTorch.
Read Moretl;dr: A GitHub gist with commands walk-through is available here.
Read MoreI've been evolving my productivity process for a long time and there are a few aspects that really work for me. One of them is doing official starting and ending ceremonies. By official I mean a session where you pretend to report findings to your boss although they and everyone else are (suspiciously) quite. Often that report means going through the project template and discussing all notes. Then, ceremonially move the project file/note to a different directory with all other completed (not necessarily successfully) projects and writing out learnings.
Read Moretl;dr: PyEMD documentation has a section on speeding up tweaks.
Read MorePresenting to you one of the side projects: the Personal Progress.
Read MoreYep, it's one of those titles where unless you know what it's about you probably don't know what it's about. If you feel with these abbreviations, go ahead and scroll down a bit. Otherwise, let me go ahead and expand here a bit: Amazon Web Services' Cloud Development Kit for Elastic Container Service Fargate. That's probably not clearer so on to the descriptions we go.
Read MoreIn the software world, there's a saying "if you want users to get excited about a new release, change buttons' colours." (I'm saying this.) Go ahead, look around; looks different, right?
Read MoreDespite my strong opinion against the AWS Glue with its unfriendly documentation and strange approach to anything... I ended up using it as the main framework for an AWS native ETL (Extract, Transform, Load) service. The whole journey felt like trying to make divorced parents get back together. They're working together but the process felt artificial and not sure whether they're meant for each other. The success was due to finding out Glue's some dirty secrets.
Read MoreMost of my projects now are separated from the beginning into logical entities and put into docker. This makes the deployment and replacement much easier to manage but the development many rituals comands.
Read MoreI've been postponing describing my experience with AWS Glue for a couple of months now. It's one of those things that I really wanted to get out of myself but it hurts to even speak up. Let's end this pain, let's end this now. Ehkem... AWS Glue sucks.
Read MoreLet me share with you the thing I found on the internet. That thing is called Exist.io. They aren't paying me for this ad but I'm still going to do it as it reignited my push for the quantified self.
Read MoreThere has been a surprising shift in my approach to writing. Any external request to express or clarify something in a written for would evoke a passionate hatred. I could feel the amount of time that is going to be wasted for no good reason what-so-ever. What others might complete in 5 min I would need to spend more than an hour. The reason was that many people considered my expression style as "weird" or "unusual", and would often ask for clarifications in the way they wanted just to make sure that we are on the same page. Talking is easier because you quickly clarify these concerns and carry on. Writing, however, is a double pain. I'd spend hours on rewiring and polishing a couple of sentences trying to clarify what might be potentially unclear and, then, I'd spend a couple on replying to feedback which typically meant "unclear." It's a frustration spiral.
Read MoreHaving more time I started retrospect on my previous projects and attempt to extract characteristics that lead either to success, or failure. As one might expect, these characteristic change in time. Currently, I’m doing better off dividing work into chunks and deal one at the time but a while back it was easier to just get into working and the immediate task would pop out. Going forward my habits and motivation will definitely change again so it’s difficult to impose a strict ‘successful project’ template, but, we aren’t setting things in stone and if needed we’ll change them.
Read MoreWorking for a tech behemoth corp is significantly different than what I expected joining. There is plenty of corpo bullshit, "vapor-ware" and ass-lickers that simply want to game out the system to climb the ladder. But, obviously, that's not all it has. Can't tell how is everywhere, but where I am, I must admit that I'm surrounded mainly by competent people and well-matured ideas.
Read MoreThere was a significant update on my title. Since the end of November, I am officially a PhD. The relief is immense. Obviously, life goes on and nothing has significantly changed on the outside but I can see that my approach to things lighten up and the approach of "Yes can do" returned. I'm open to new projects and ideas.
Read MoreIn Ubuntu 18.04, typing in Nautilus file manager makes a deep search through it's all subdirectories. This is potentially a powerful and convenient tool but, unfortunately, not for me as it doesn't fit into my workflow. I'm less of the GUI fan and only use a graphical file manager when I need to do some quick drag and drops on file bulks. Typically, if I'm using Nautilus, I'm already in the directory where I need to be and just want to quickly jump to a file/dir by the name. Shortly: yes, bring back the ye olde stuff.
Read MoreThe last post about AWS EMR and S3 has resulted in a few people messaging me directly. To ease others let me add something about how I approach a specific problem.
Read MoreOne of the recent projects I've worked on involved processing billions of row stored in AWS S3 in terabyte size data. That was the biggest I've worked so far and, even though I don't like the term, it broke through the big data barrier. Just handling the data with popular toolkits, such as scikit-learn or MXNet, created so many problems that it was easier to create our own solutions. But the biggest surprises came from places least expected.
Read MoreAlthough learning and book knowledge are the best, my personal relationship with reading activity is not the friendliest. Being focused on the text is a huge struggle and I often need to re-read sentences to actually read it. That's why sometimes I use text-to-speech (TTS) software or service.
Read MoreQuick ad hoc command:
:%!python -m json.toolFor reusing and typing more update your `.vimrc` file with:
com! FormatJSON %!python -m json.toolAfter this you should be able to use :FormatJSON command. Read More
Chances are you heard that multiprocessing in Python is hard. That it takes time and, actually, don't even try because there's something like global interpreter lock (GIL), so it isn't even true parallel execution. Well, GIL is true, but the rest is a lie. Multiprocessing in Python is rather easy.
Read MoreCEEMDAN is available in Python through PyEMD.
There are as many empirical mode decomposition (EMD) variations as many teams are working on it. Everyone notices that in general EMD is very helpful method, yet, there's room for improvement. Few years back I have stopped doing modifications myself in exchange for working on mathematically sound model of coupled oscillator. Nevertheless, since I spent quite a lot of time on EMDs and have enjoy playing with it, from time to time something will catch my eye. This is what happened with Complete Ensemble EMD with Adaptive Noise (CEEMDAN).
Read MoreMy "writing" work currently goes somewhere else and have little motivation to write anything here. But, there's something that only internet can help, whether that's through actual help or simply transferring my annoyance.
Read MoreAmazon with it's Amazon Web Service (AWS) is pretty cool. It gives you access to remote machine which you don't have to maintain. Actually you don't have to do anything other than use it. All machines come in different flavours, but what tastes better than free? Granted that it's extremely limited, but surely we can squeeze something out of it. Right?
Read MoreOne of the packages I intend long term maintain and support is Python implementation of Empirical Mode Decomposition (EMD) called PyEMD. I will skip introduction of the method as it has been explained in few other posts [1, 2, 3, ...]. This blog entry is more about announcement of new feature which also means new version.
Read MoreI'm becoming increasingly convinced that using GitHub to share projects, even in work-in-progress (WIP) state is quite beneficial. For one, someone can quickly point out that such project exists or suggest a tool which would simplify some operations. What's more, with tools like Travis-CI or CodeCov it's easy to follow whether the project is actually buildable or whether updates broke functionally of other features. Real convenience! Although, there is some price to pay for these goods.
Read Moretl;dr: Project on github: https://github.com/laszukdawid/pystan-kuramoto
Read MoreOnly recently I've started using VIM as my default editor and... I'm amazed.
I'm not advocating here for its superiority over any other text editor. You might be into design or simply wish for a graphical interface. I'm into not-using-mouse too much, so it actually fits quite well my purposes. Having said that, it took me a while to discover it with more commands than saving and exiting.
Read MoreThose who know me, know that I'm not a fan of using mouse. Whenever possible I try to avoid using it. Although, after years of experience, I am fluent in keyboarding there are still some tasks that I need mouse. Well, until quite recent.
The number one of annoying tasks is copying things from terminal to clipboard. Depending what exactly I needed to do with it I'd either select something with mouse and then copy, or stream output to file and then select it within editor. Some people found it weird, but yes, often copying through editor is much faster.
Read MoreCode for Kuramoto in Python is available here or from code subpage. Explanation on how to use it is on the bottom of this post.
Read MoreIt came to my attention that my PSO for Python is actually quite popular. Only after few people contacted me I've noticed that the public version was not the same that I've been using. It wasn't bad, but definitely not as good. Thus, obviously, I've updated the version and cleaned it a bit.
Read MoreRecent challenge I had was to convert two given numpy arrays such it'll be possible to have them in common type without losing information. One can follow this link for stackoverflow entry.
Read MoreSometimes when we ask for random we don't actually mean random by just random. Yes, pseudo-random.
Consider unitary distribution with ranges 0 and 1. Say you want to draw 5 samples. Selecting them at random would mean that we might end up with set of {0, 0.1, 0.02, 0.09, 0.01} or {0.11, 0.99, 0.09, 0.91, 0.01}. Yes, these values don't seem very random, but that's the thing about randomness, that it randomly can seem to not be random.
Read MoreOnly recently I have started to use Python 3. It's been out for good 8+ years and all these excuses about incompatibility with some packages were just lazy. Most packages I use are already ported and if I ever find that something is incompatible... well, I'll think then. But for now let me pat myself on the back for this great leap, because:
Read MoreA while ago I've started to taste a bit how it feels to work in industry and it feels quite nice. Maybe that's the specificity of field projected onto the industry, or being tired of how academia works, but I'm enjoying extremely learning all the details about Computer Science, programming and newest technologies.
Read MoreI'm an avid podcast listener. Whenever there's something that only requires sight or not much focus I'll try to do it with my headphones on. Great thing about podcasts is that they they are more up-to-date than audiobooks and have reasonably short lengths, so there's always a fit.
Read Moretl;dr: Read subject and click on link below.
Some of you, i.e. those lucky ones with connection to the outside World, are probably aware about Machine Learning community trying to aggressively change our lives for better. Regardless whether we like it or not, they're doing it. Some do this for money, others for fame, and those wicked ones just for fun.
Read MoreThis blog needs some lightening up and what a better way to do that than post some graphs? Exactly! I like to collect data on various things and then make a graph of them. Things are much better when presented with axis and some number around it.
Read Moretl;dr: I'm going less quality, more quantity.
I like to write. I write lots, but usually don't publish it. Whenever I think about something and write it down, I then rethink what I thought and want to rewrite what I wrote. It takes me ages to write something very precise and something that I wouldn't be immediately ashamed of. This blog was meant to be the place for that content, for things I wouldn't quickly regret. And, although, I'm rather OK with the content, I'm deeply disappointed with the frequency. Thus: change.
Read MoreAnother of my papers [1] (see About) has been published recently. Unfortunately, I cannot present it here in a full scope. Below I'm presenting an abstract from the paper and invite interested people in contacting me. This topic is really interesting and there is plenty to be done.
Read MoreFor the past few years I've been teaching few modules for Master students, such as programming or signal processing. In most of them, one of the main and important tasks is to write a data buffer. Usual requirement is to be able to return last N samples or T seconds. However, conceptually most of students understands, implementing this seems to be quite difficult. But it is not. Unfortunately their first line of action, rather than think themselves, is to search for the solution on the Internet. There are many attempts, better or worse, but since they are looking for it, I could also provide with my own solution. Hopefully it helps others as well.
Read MoreFor a while now, I've been involved in Kuramoto models and adjusting them to data. Recently I've stumbled upon using Bayesian inference on predicting parameters in time-evolving dynamics. Team from Physics Department at University of Lancaster has produced many papers, but to provide with some reference it might be worth to look at [1].
Read More[Updated version available. See newest post.]
tl;dr: For Python PSO code head to codes subpage.
Read MoreSearch