Category Archives: Software

Software Things

Join MJPEG videos from a CritterCam

Join MJPEG videos from a CritterCam
Ingredients: Intermediate I.T, Tinker-comfort, Google, and 
latest ffmpeg, Linux etc, Sample at End.

ffmpeg is command line only tool which may seem weird for video but, hold up- a lot of video tasks like the following lend themselves really well to scripted heavy lifting- as here. There are literally hundreds of options. The heavyweights- YouTube, Facebook etc use ffmpeg to transcode and process stuff automagically– No GUI for them!.. It is a major & deceptively powerful player.

Critter-cams- you know, the ones that can record short video clips of  daily+nightly beasties when motion is sensed?
How to join these video clips? Same story for car DVRs- AKA DashCams- those devices routinely produce a large array of short video files,
often non-contiguous in timing. I assume there is No Audio – Yet.
Not sure what happens if there IS : )

I thought it would be fun to join them together after discovering the movie video format was MJPEG also known as motion JPEG.
it’s not particularly well compressed because the video only contains JPEG frames that are themselves compressed but there is no optimisation between adjacent frames. It does make it easy to split out single JPEGs or join them without loss and do other clever tricks like the following one. Join a whole lot together! How?

First of all– check that the clips really ARE in MJPEG format, 
This article will only describe how to join such simple files ‘natively’.

‘Real’ Video Files like h.264 & HEVC/h.265 require re encoding–
MJPEG format as here does not though, is simple,
and goes very quickly if done right. Let’s first of all check a single clip to see how it’s made by using ffmpeg without any output file specified:

$ffmpeg -hide_banner -i MFDC1354.AVI
Input #0, avi, from 'MFDC1354.AVI':
Duration: 00:00:10.00, start: 0.000000, bitrate: 3839 kb/s
Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 640x480, 3867 kb/s, 12.50 fps, 12.50 tbr, 12.50 tbn
At least one output file must be specified

This is using ffmpeg without any output specified-
so this single clip  is a 10s one in MJPEG format, at 640×480 resolution Good!
I have eighty-nine such AVI files in one folder from about 2 weeks.
Let’s get ffmpeg to join ’em in sequence.
Rather clunkily, we need a separate text-file ‘running order’ telling ffmpeg which files to join, let’s call it list.txt; It’s expected to contain one file path per line- there woudl be 89 lines.  like:
file “FILE1.AVI”
file “FILE2.AVI”
you get the idea… Let’s automate that:

for f in *.AVI; do echo "file '$f'" >> list.txt; done

The list by default appears to be in  chronological file time stamp order, so be careful NOT to touch the create/modify date too much:)
In Case You Wonder:
$cat *.AVI >> out.avi 
will NOT work as each single file has ‘container’ data
(AVI in this case) that would also get jammed into the output where only the 1st 10 second header would be ‘seen’ and might just get played the first 10 seconds back, skipping the remaining 88 clips which it cannot ‘see’ as such : )

So. now Tell ffmpeg What to

$ffmpeg -f concat -safe 0 -i list.txt -c copy all.avi

We now have a far larger MJPEG file with all parts joined and the size being the sum of the parts, and No Music.
Do try Playing the (silent?) output  Back before doing the Next Thing:

$ffmpeg -i all.avi -i music.m4a  -shortest -c:v libx264 -pix_fmt yuv420p All_WithMusic_h264.mp4

This produces a highly compatible video file with decent compression, in my case 10x smaller with no discernible loss in quality.
 You might try “-c:v libx265” rather than h264 for super compressed HEVC video, in my case, 20x smaller!
But it will be 4-5 times slower and may not be compatible with HTML5 Web Streaming.
The “-pix_fmt yuv420p” I cannot easily explain, except that there are weird color artifacts if you do not add that.
The “-i music.m4a” does the expected thing- adds a music-track, which must, not surprisingly, exist with the right name somewhere : )
“-shortest” will stop the video AND audio  should the audio  be longer, otherwise audio will keep playing if it’s longer than the video with the last video frame frozen : ).
Some google searches seem to suggest a far more complex command line- Latest ffmpeg seems to understand the defaults things you mean to do, in the given order though,
and worked for me

My music happens to be in “aac” format- other types could be used, it will depend on what program will be used to play your video+audio back;
ideally of course avoid uncompressed audio formats like WAV.
Squash ’em nicely, first. : )
I find with Some Delight that the Music Metatdata- Artist, title etc may also get copied and exist within the output  file where tools like “mediainfo” can ‘see’ it.

I test my videos with wonderful VLC, by the way,
But Here it is through web based HTML5:

Moultrie Crittercam concatenated eighty-nine 10 second MJPEG clips over about 2 weeks into one h.264 Video 04:46  long compressed video with Music : ).
The icon at lower left is the moon-phase that this Critter Cam seems to know about : )

  • End.

HEIF, HEIC new Image FileFormat

Out of the blue almost, Well, about 2015 in fact,
it seems, came a New Image File Format:
“High Efficiency Image format” .heif & .heic
that miraculously is a third or even a half the file size of conventional JPG and GIF formats we are so used to with comparable-or even better- image quality :
It also ‘does’ a lot more and has more capabilities

Starting with  Phone 7 with IOS 11 or newer,  and contemporary  Android devices, this is the default format– and No Wonder. The ‘high efficiency’ achieved would normally be at the cost of heat and precious battery life  in portable devices due to the heavyweight software calculations involved.
But now, this process is embedded within the Silicon Chips  your device for far higher efficiency- Hardware. Tis is also generally true of Intel CPUs of ix 8th generation or higher (Since 2015 approx) 
More Here:

https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format

A comparable format for Video appeared about the same time, HEVC: High Efficiency VIDEO coding (more about that elsewhere) suffice to say I noticed without knowing why, a 50% DROP in  Face-time Bandwidth used if both ends  had at least an iPhone 7 or above :))

So your phone’s precious Storage is spared when taking still images- Or videos- Transferring them uses less of the main limiting resource on the Internet–Bandwidth, and everyone is happy, right?

Almost.
Built in to most newer devices is the ability to ‘down convert’ if sending to older formats. I assume SOME negotiation takes place but it’s relatively easy to convert formats ‘transparently, on-the-fly’ using the Silicon Chips I just mentioned, 
They are handed the job and hand it back Done.
The processing MUST be real time- albeit with some acceptable  latency-  else Converting, Streaming, Saving would impose unacceptable delays.
Minimal Buffering/Latency  is a side effect of Powerful Silicon.

How about we Linux users? I use Linux MINT TARA, based on Ubuntu 19 something,  all nice and up to date, but with a 10 year old CPU and no special graphics card GPU to help..;
No real-time stuff for me!
And sure enough no Thumbnail Displays of  .HEIF images in its native File Manager NEMO  nor edit them in Gimp, the “Graphical Image File Manipulation Program”, despite its name
etc. 

sudo apt-get  install libheif-examples
the command : “apropos heif”

Shows you now have:

heif-convert (1) – convert HEIC/HEIF image
heif-enc (1) – convert image to HEIC/HEIF
heif-info (1) – show information on HEIC/HEIF file
heif-thumbnailer (1) – create thumbnails from HEIC/HEIF files

Fun! It works, too. I shall Leave The Examples Of Batch Processing Up To The CLI Scripters : )

 

.. To be Continued …

 

 

For command line tools, add:

El Capitan MAC Install Error

A mid 2009 Macbook, launching El Capitan (OS 10.11.x) install from a bootable flash drive. Newer MAC OSX will not work on this older MAC. InstallĀ  gets most of the way then mysteriously complains:
no packages were eligible for install.

Also mysterious errors when trying Command-R Recovery Partition re installs.
Remade bootable Flash drives (Instructions elsewhere) and much more, many Trials and Errors.
I DID notice that the re install seemed to require Internet Access and did on couple occasions actually want an Apple ID for some reason during the re install. the conditions seemed to vary due to unknown factors but at one point I saw
“Checking Eligibility with Apple”

I later found that just like the IOS on iPhones, etc, a check-in with the Apple Mothership is performed, and some sort of Certificate is used to verify I really have an Apple, I suppose..
Someone out there noted there is a DATE problem with Apple’s Certificate- Or maybe the one in the El Capitan software.
The Workaround was to go into Utilities– terminal during the stage right before the files start to Install and using the Terminal Date command to FAKE A DATE WHEN El Capitan would have been considered Current Software. (I am guessing here)
So Here Is What Worked
date 0418171516
the ORIGINAL ARTICLE is here as well as many others that allude to this issue. You then need to RE BOOT and try again (Date should NOT updated despite Internet Access, but CHECK)
Depending on which particular version of the Apple El Capitan installer you use. the date may vary but it was originally released September 30, 2015 with half a dozen incremental updates.
No Luck? Mess further with the Date!
Once things are ‘happily’ Booted from the local hard drive is shoudl be possible to set the date normally.
Good Luck. your Comments are welcomed!