The Personal Progress – opinion tracker

Presenting to you one of the side projects: the Personal Progress.

The goal of the project is to track over long time personal opinions on everything. As we change our social surroundings, we meet new people, we learn about new experiences and we discover things that we haven’t known before. Each event has an impact on us; some will confirm our believes and others will challenge them. The personality changes in slowly but gradually, making it difficult to observe on the day-to-day basis. It’s typically more obvious to people with whom we don’t interact too often. Why leave this entertainment only to them? Go ahead and leave breadcrumbs for yourself.

Unfortunately, it’s difficult to learn about the past yourself unless the Past You allowed for it. Blogs and diaries definitely help in this but their content is typically written in a form that makes it generally difficult to extract specific opinion. The Personal Progress asks direct question and reminds itself every so-often to check for any update.

The result will be achived over years. On top of simple, side-by-side answers’ comparison each question will be classified into broad domains. You’ll be able to check how they group together. Each answer will also be quantified on sentiment and category-specific axes allowing to extract intensity value.

Feel free to give it a go and let me know whether you (dis)like it, you think something requires improvement or simply as for additional features. The tool is still a work in progress but it’s usable and I’m using it.

Refreshment. Ain’t no scientist no more.

In the software world, there’s a saying “if you want users to get excited about a new release, change buttons’ colours.” (I’m saying this.) Go ahead, look around; looks different, right?

As mentioned in previous notes, I want to write more often. Actually, I do write more often but not on this blog. Diving on “why” it seems that there are high expectations on the quality. It isn’t about wording or grammar but rather the feeling that I need to write something science-related. To be fair, not only “science” but either I have contributed to it or have some thoughts. Given that my primary income in the past few years came from making computers do magic, let’s just put it out there: “I ain’t no scientist no more”…

… and now, quickly, clarify, before anyone leaves. I am a scientist; I do feel like one. I’m going to why-analyze every aspect of mundane activity until it reaches either atoms or astral powers. If anyone asks or doubts I’m pulling official email from the Uni. But, that’s not to say that’s my main occupation. Same as some “powerful” people will dumb-down themselves by claiming they’re “Father. Husband. The president of the United States.”, I’m the “Average human. Scientist. A person who uses computers daily be that for creating functional services or performing data manipulations.”

This announcement(?) this needs to come now since I’m getting close to the point when I can start sharing some things I’ve been thinking about and/or worked on. Needless to say, I’ll need some audience to share these ideas/projects with. I won’t go into “please like and subscribe” but if you want to share posts or tell others about any of my work you have my blessing.

What are the changes? Landing page gets refreshment – checked. Static pages with ongoing thoughts and projects. Blog posts on random topics but hopefully in theme as that’s going to be on what I’m working, which hopefully is in theme.

There. Typical blog post on how I’m going to write more blog posts. Excited?! mumble mumble…

Barren pages get some words raining

There has been a surprising shift in my approach to writing. Any external request to express or clarify something in a written for would evoke a passionate hatred. I could feel the amount of time that is going to be wasted for no good reason what-so-ever. What others might complete in 5 min I would need to spend more than an hour. The reason was that many people considered my expression style as “weird” or “unusual”, and would often ask for clarifications in the way they wanted just to make sure that we are on the same page. Talking is easier because you quickly clarify these concerns and carry on. Writing, however, is a double pain. I’d spend hours on rewiring and polishing a couple of sentences trying to clarify what might be potentially unclear and, then, I’d spend a couple on replying to feedback which typically meant “unclear.” It’s a frustration spiral.

At times when I’d panic and plea for help, the most common advice was reading more books. Probably great advice, but doesn’t work for me. As an (too) active person, I have trouble sitting down and reading. The most productive reading is when I’m walking in a dull room with white noise and without anyone. As you might expect, this isn’t always possible, so my reading is slow. How can you write clearly if you don’t know the rules, and how do you know the rules if school grammar is different from the common folks’ expectation?

The solution isn’t clear but I think I’m approaching it. Not only starting a new paragraph isn’t problematic anymore, but I’m also spending less time on polishing expressed thoughts. A year ago I’d quickly run away from any writing task but now I’m scribbling a coupled of pages every day. Some pages are for myself; personal notes and thoughts to keep my life organized and properly archived. Others are for work. Even though, at the moment, I’m a Software Developer Engineer, which some might imagine solely writing code, I’m daily updating documents, be that a documentation, design explanation or ideas pitching.

What’s the change? To be honest, I’m not sure and it might be anything. But, if I were to highlight a few potential reasons, three that come to mind are:

  1. The Elements of Style” by Strunk & White. It took me a while to start reading this book but the delay was mainly because “it is a book” and books are scary as they typically have more than 50 pages (too much). But, depending on the edition, it’s about 30 pages long and concisely explains “dos and don’ts” in writing. Logical rules to follow; a simple guideline to follow; a blessing!
  2. Grammarly. A service and a web application acting a spell-checker on steroids. It detects grammar misuses, unclear sentences, word repetitions and has a built-in thesaurus. Each detection comes with a brief explanation of why it was triggered and how to solve the issue. Given the broad demand and lack of alternatives, there’s definitely more to come from the Grammarly.
  3. Lower expectations. Quality bar for writing outside academia is significantly lower. This isn’t an insult. It’s actually great to not stress out about imperfect sentences and slightly ambiguous words. Words don’t carry that much liability so people are encouraged to make mistakes are that can be cleaned in the process. The focus is more on the story progression and less on defending thoughts. Lower threshold allows realistically learning from experience.

This blog as a whole was intended to be scientific. Even when I wrote that it’ll be less ‘scientific’ than before, I meant that it’d be less ‘precise’ in words’ synthesis. Given that I already left academia and my ties with the scientific side are decreasing, the intention needs to change. Expect more docs on general topics but still in engineering/technology theme. I’m still considering having a separate blog dedicated to philosophy, so, who knows?

Project template

Having more time I started retrospect on my previous projects and attempt to extract characteristics that lead either to success, or failure. As one might expect, these characteristic change in time. Currently, I’m doing better off dividing work into chunks and deal one at the time but a while back it was easier to just get into working and the immediate task would pop out. Going forward my habits and motivation will definitely change again so it’s difficult to impose a strict ‘successful project’ template, but, we aren’t setting things in stone and if needed we’ll change them.

Analysing notes from the last 10 years I noticed that there were a few common parts which, if defined properly allowed me to succeed in the project or fail gracefully. I use “properly” because sometimes I tried to cheat myself expecting that future me might forget and will be fooled by the writing. However, the only thing that I managed to forget was that the “forgetting” part doesn’t work. Even now I have a vivid experience of changing the expectations for each project but had to think a bit why I did that. Hopefully I wiser.

Now, whenever I’m thinking about starting a new project it goes through a few phases.

Project phases

Phase 0

Quickly write a sentence or two about the project. It’s usually a sentence on what the idea is and how I came to think about it. This typically allows me to let that thought go for a while and continue on whatever I was doing before. For such quick notes I’m using Google Keep – two taps to input note – but I’m looking to escape Google. Once it’s written I can let it mature for some time, typically until the next evening.

Phase 1

Note: If the project is super short or it doesn’t make sense to start it in a few weeks, I’ll skip over this phase or delay it.

Typically a day delay and a full night rest allow me to validate whether it makes sense to devote some time to the idea. In this phase I’ll open a text editor and will try to write at least 100 words on how I see this idea to grow and what would it mean to do it. Writing out prevents me from cheating myself with the optimism of the moment. It’s a bit tedious to write on something in my head sounds awesome but the logic is if I can’t be bothered to spend 5 min writing out on the project it’s unlikely I’m going to want to work on it.

Phase 2

Create a document to monitor the progress of the project and populate the top with the following template:


{Project Name}

Success: {The definition of successfully finished project.}
The **Why** statement: {Why am I doing this project?}
Expected length: Coarse: {Long/mid/short term}
Fine: {XX weeks/months}
Health check: {Define period of time to check on the project.}

The goal is to check these properties every {health check} period and make sure that they’re still valid. This task won’t be successful unless the table is easy to read and short. The rest of the document should be devoted to progress updates. Depending on the project this can be in form of a timeline with timestamps, or living document. I’ve started using the http://notion.so/ for this; it’s a collaborative, realtime doc with plenty of key shortcuts and markdown enabled.

Phase 3

This phase is about execution. Every {health check} review priority of the project and set aside some time for it. If it happens that for a few {health checks} you weren’t able to do anything related, go ahead and update the project. I like to keep track of ongoing projects in a special Trello board and create individual tasks to “daily” board. I could’ve used Notion for it as well, but I find Trello awesome for day-to-day tasks and it’s easy to accidentally check what’s going on when you’re a click away with a constant reminder.

It’s just a phase

There’s no point in cheating yourself that you’re going to do something when you don’t want to. Almost everyone I talked to had experienced the inability to do anything caused by mental overload with things that had to be done. I’m assuming that the driver for these projects wasn’t a necessity but the choice to do something awesome in spare time. If that’s the case feel free to let some go. That’s difficult due to the sunk cost fallacy but the goal of this template is to help rationalise the decision. Our values and motivation change, and so should our priorities. (It won’t be probably long before I feel stupid for ever writing this blog post.)

JSON pretty formatting in vim

Quick ad hoc command:

:%!python -m json.tool

For reusing and typing more update your `.vimrc` file with:

com! FormatJSON %!python -m json.tool

After this you should be able to use :FormatJSON command.

This would convert:

[{"classification":"emrfs-site", "properties":{"fs.s3.consistent.retryPeriodSeconds":"10", "fs.s3.consistent":"true", "fs.s3.consistent.retryCount":"5", "fs.s3.consistent.metadata.tableName":"EmrFSMetadata"}, "configurations":[]},{"classification":"spark", "properties":{"maximizeResourceAllocation":"true"}, "configurations":[]},{"classification":"spark-env", "properties":{}, "configurations":[{"classification":"export", "properties":{"PYSPARK_PYTHON":"/usr/bin/python3"}, "configurations":[]}]}]

into

[
    {
        "classification": "emrfs-site",
        "configurations": [],
        "properties": {
            "fs.s3.consistent": "true",
            "fs.s3.consistent.metadata.tableName": "EmrFSMetadata",
            "fs.s3.consistent.retryCount": "5",
            "fs.s3.consistent.retryPeriodSeconds": "10"
        }
    },
    {
        "classification": "spark",
        "configurations": [],
        "properties": {
            "maximizeResourceAllocation": "true"
        }
    },
    {
        "classification": "spark-env",
        "configurations": [
            {
                "classification": "export",
                "configurations": [],
                "properties": {
                    "PYSPARK_PYTHON": "/usr/bin/python3"
                }
            }
        ],
        "properties": {}
    }
]

Take it easy

tl;dr: I’m going less quality, more quantity.

I like to write. I write lots, but usually don’t publish it. Whenever I think about something and write it down, I then rethink what I thought and want to rewrite what I wrote. It takes me ages to write something very precise and something that I wouldn’t be immediately ashamed of. This blog was meant to be the place for that content, for things I wouldn’t quickly regret. And, although, I’m rather OK with the content, I’m deeply disappointed with the frequency. Thus: change.

This blog initially was meant to be my academical window. Something that when people look me up (for whatever reason), they’d see something over which I would have control. However, my view on World has yet again changed. Despite my passion for research and likeness for academia free-thinking diverse environment, I’ve decided to leave it. There is so much great research being done in industry and private sector that I think I’m shooting myself in foot by sticking only to that audience.

Plan for this blog is changing to present cool things that happen and fun stuff I’m working on. I enjoy working and I like to write. If the content is not up to standard quality, shame. But not having a content at all is just disgraceful. How can I show people what I’m working on, if I don’t show them anything at all.

In Python’s mantra: It’s easier to ask for forgiveness than for permission.