Today I decided to apply my little knowledge of Bash scripting and automate a task I perform routinely for my Markets project. What is also interesting is the fact I am testing my DIY Acoustic Box for my Microphone (Image Below). Do share this article and if you have questions or suggestions, feel free to add that to the comment below.
I have been using Linux for the past 10 years and will in no way consider myself a Power User. I can do a thing or two, I have a little amount of system administration skills and I am not afraid of the terminal. Running a few commands here and there is something every Linux enthusiast is comfortable doing. Being a Software developer for a couple of years, I am usually intrigued by anything that looks cryptic and feels like programming and so the terminal plays a big part in my everyday existence.
Recently, I have been working a lot with the Bash scripting language/tool. My bash programming isn’t that legendary but with the help of Google and Stackoverflow, I can easily look up syntaxes for Bash statements and eventually get my intended task done. I recently had a task in the office that required me to work with a 200MB+ sized file. CSV files are the simplest file format you get to work with in most programming environments. Usually when you want to generate a report that can easily be opened in a spreadsheet application and also easily consumed by a machine.
I have been itching to write a blog article but haven’t recovered from my writer’s block just yet. So I thought maybe instead I should post a video. So in this article, I post of video introducing a favourite tool of mine called GitKraken. Gitkraken is a GUI told built to make working with your Git Repository a breeze. After I adopted GitKraken, I don’t see myself spending that much time on the command line unless I have to.
I hope you enjoy this video and feel free to leave a comment.
It can be fun being a developer, but sometimes you can forget you are one and begin to think human. Today as been a somewhat long day from visiting my GP and ending up in the hospital behind a long queue of patients waiting for an X-ray. I am at my desk with a couple of hours to churn through, I decided to tackle a long pending Ticket. An hour and a few minutes later, I am done with the update but there is a second part to this ticket. I need to refresh a lot of data in a CSV file stored remotely on an AWS S3 bucket.
I download the file and decide to tweak the refresh locally. So I update the code and statically type in date range parameter. I was meant to get a year’s worth of data. We looking at some heavy MegaByte worth of JSON data. We do agree that is just suicidal to think you can transfer 80MB worth of JSON data over the wire from one HTTP request. So the task would be to break the API date range between the months of the year.
I tried breaking the request down but got a 400 – Bad Request error message. So as usual, I decided to test the boundaries of the API by reducing my range from 1 year to Monthly, Weekly and then I came upon the 1 Day interval as the ideal range to work with. So I started out by manually adjusting the data and seeing what data gets pulled. With a bit of frustration and trying to figure out how I can automate the changing of the API date range. For some reason, the thought writing new code to get the result didn’t come to me on time for some reason. I spoke to my colleague next to me, and he suggested I write a function to do all the date range update. This was a brilliant idea, I could write this code, get the effect I wanted and revert code back. We can discuss code review and why date range was hard-coded in another post.
I wonder why I didn’t think of this. Sometimes as developers, we do miss the joy of innovating, writing new codes and being innovative with solutions to a simple task. I did write the code as expected, got the result I wanted but still came across other bottlenecks which had more to do with the API than my code.
Morale of the story:
Never forget you are a developer. Always seek to innovate.
If you consume a lot of Internet resources, we keep a record of pages we visited or hope to re-visit at a later date. Modern day browsers provide a feature called Bookmarks. Bookmarks stores links to pages or articles you found interesting and you wish to revisit another time. Working with bookmarks on your local machine is the easy part, the trouble comes when you use different machines during the course of your week (Laptop, Desktop, Tablet, Phones).