Show transfer percentage and speed when copying files in Linux and Bash As great as Linux/Bash is, sometimes I wish some of the built-in commands did a little bit more than the bare minimum. Of these the 'cp' command is something that I always thought should have had a way to show progress when copying files. There are many questions about this very thing on StackExchange like this one and this one. My preferred solution is to use 'rsync' but I'm also a big fan of simplifying things and can never remember all the various command line options required. So this post shows my little trick to enabling file copy speed and percentage display when copying files.
Prevent errors during ffmpeg execution in a loop I was using my Odroid XU4 based media server to transcode some videos using ffmpeg when I came across some strange errors appearing on the console. When I would run ffmpeg as a single command, it would work, however running it in a loop presented errors (or rather strange behaviour).
Continuous frame updates added to Trafficr and ExpressWayCam.com About a month ago I made my Trafficr set of scripts available on GitHub and wrote a short intro about setting them up. The first version of those scripts worked pretty well but they were missing continuous frame updates. That meant that the web page would have to be manually refreshed to get updated cam frames. This version addresses that gap.
Set up a time lapse webcam with Trafficr I've written previously about setting up a traffic webcam on OS X, well now I decided to move away from my old OS X Lion Server so couldn't use those scripts any more. This past weekend I took some time to rewrite my cam scripts to work on Linux and this is what I ended up with...
Display OdroidXU4 system status on Odroid-Show2 Since the DietPi-CloudShell doesn't work properly on the Odroid-Show2, I decided to make my own script to display the system status on the Odroid-Show2. I reused some of the the code from DietPi-CloudShell for this purpose.
Compiling unrar on DietPi for ARM I've recently had to build the 'unrar' utility on my Odroid XU4 after installing DietPi on it. I know it is available on the non-free repository and can be installed using apt-get, but that didn't seem to have the latest version so I decided to build it from source instead. Turns out it's quite easy.
Running test case in SoapUI fails to run test suite Groovy Setup Script I've been doing testing with SoapUI testrunner.sh script recently and have come across a very peculiar behaviour. If you run a specific test case, testrunner.sh will not execute the Groovy Setup Script for the test suite that the test case belongs to. However at the same time if you enable response capture and don't specify the test suite, the test suite name will appear as a part of the request/response file name. It seems that SoapUI is still processing the test suite, just not running any of its setup code.
Testing using SoapUI property values, SOAP response capture and no GUI I recently had to analyse the performance of a web service that can be used to upload documents into a database, SoapUI was the natural choice for this but I didn't want to run it from my laptop so I could avoid network transfer times interfering with my results. This meant I had to run SoapUI directly on the server where the web service was running.
Digitizing old photos with a scanner and some handy scripts Over the past couple of months I've been digitizing my parents' old photos with the aid of a scanner and some ImageMagick scripts. The scanning itself is quite an arduous process, and the cropping and rotating of the images is equally as much work so I decided to see if I could automate at least a part of this.
Preventing SSH client disconnection by remote host A lot of work gets done using SSH and one of the most frustrating things to happen is when the remote server disconnects you due to inactivity, but while you had your environment set up just the right way for the tasks you were doing. Imagine that you had to have a number of environment variables set just right in your remote shell and you stepped out for lunch only to come back to this...
Java Mission Control requires GTK but which version? I've recently had one of the DBAs on a project ask me what GTK version is required to run Java Mission Control with JDK 7. I thought easy, the documentation should tell me this, but after searching for several hours I've come up with nothing, apart from 'GTK Libraries Required' as stated in the Oracle Java Mission Control 5.3 Certified System Configurations.
Generate XML Schema documentation the easy way with XS3P I have to work with the CDA XML schema on a regular basis and sometimes it is easier to check things either via a tool that can visualise the schema or through hyperlinked documentation that describes the schema in a concise way. Unfortunately, as I found out, most tools can't handle a complex schema that has multiple cyclic references within it. So what's the solution? Use a tool that doesn't analyse the XSD! This is where xs3p comes in.
Running a PHP script at scheduled times on a GoDaddy hosting account with cron I had a need to run a certain PHP script on a regular basis as a part of my blog maintenance. Since GoDaddy uses the more-or-less standard cPanel which includes a Cron Job scheduling option, making this happen was fairly trivial. This is a quick guide to show how to schedule a PHP script to run automatically at regular intervals.
Adding swap space on a RaspberryPi running Minibian My photo frame has had some new pictures added to it recently after which it started to crash. Turns out it was running out of memory. I guess 256Mb of RAM could not accommodate for a 128Mb JVM as well as the OS. I decided to add some virtual memory swap space to it only to find that dphys-swapfile was not installed by default in Minibian. This article describes the steps I followed to get virtual memory working.
Visualising Oracle database partition data distribution with gnuplot We use partitioning in at least one of the databases at work, changing to partitioned tables has allowed us to keep the system running within our imposed SLAs. Recently I've started to wonder however, what happens when we start hammering individual partitions more than others. The data distribution would be skewed and the benefits of partitioning lost. I decided to put together a bit of SQL and then use gnuplot to show me how well our data is distributed.
Lenovo LT1421 USB DisplayLink monitor on a RaspberryPi I started a project to build a photo frame out of a Raspberry Pi (Model A) and an old Lenovo LT1421 USB monitor that I've had laying around. There were lots of confusing posts on whether DisplayLink USB monitors work with a Raspberry Pi and to what lengths you have to go to make them work. The only option I had was to just try it for myself, so I did and these are the results. It's a lot easier than you'd think!
Change to the traffic cam script to auto-sort by year/month/day into folders As the traffic cam keeps taking photos every 5 minutes, the number of files in the archive directory quickly adds up, over the past 2 months there were around 12 thousand files. This was getting a little bit difficult to manage so I decided that it would be a good idea if the script automatically put these into directories sorted by year, month and day.
Fixing the Node remains in conflict SVN message I had the 'Node remains in conflict' error message come up recently due to an SVN update, this is how it's resolved. This error came about because I had a file in my local checkout that was not under version control, someone checked in a file with the same name into SVN. This should never have happened, the repository had its file deleted, yet every time I did an update, I got this error.
Tuning the SQL*Plus fetch size for better performance I've been doing a fair amount of work with data extraction from a database to a flat file recently using SQL*Plus and have been looking at ways to speed up the process. This is where the arraysize setting comes in handy. This setting controls how many rows are fetched from the database in a single round trip. I did some testing with various sizes and found that tweaking this value can have huge performance improvements. This is what I found with the data I was extracting.
Add a filter/search option to the OS X Terminal history command If you're like me and use the Terminal in OS X on a daily basis, you probably also use the 'history' command quite regularly too. Every time I use history I always pipe it through grep, so always, always end up typing 'history|grep blah', now isn't there an easier way? Well there is, and that's what the alias functionality is for.