Shell script ninja help needed: weekly test image downloading

June 26, 2010 – 7:44 am

Dear lazyweb: there must be a simple answer to this. I’m trying to write a shell script that a cron job can run every week to update our Sugar on a Stick (SoaS) test image repository. The ticket in question is Sugar Labs #2058. Longer explanation than usual given so those new to the dev/test/release cycle can follow along.

Basically, SoaS is a Fedora Spin, so we get nightly composes made here (as in, “Fedora automagically builds our .isos for us so we don’t have to”). In order to (we assume) save on disk space, the Fedora servers only store the latest nightly compose – once a new .iso is made, the old one is gone forever, bwahaha!

This is fantastic for developing, but not so much for testing. Expecting testers to keep up with daily builds is a bit much, and it’s putting a burden on people who are downloading them every day (possibly even getting into trouble with their ISP), so we decided to go with a weekly test cycle – each Thursday evening we’d designate the most recent image as the “image under test” and point everyone there. That way, developers would also know exactly what image people were finding bugs in each week.

Problem: in order to (we assume) save on disk space, the Fedora servers only store the latest nightly compose -  once a new .iso is made, the old  one is gone forever, bwahaha! So we need to grab the most recent image – which has a special naming – at that time and pull it down to the Sugar Labs servers so we have the files at http://download.sugarlabs.org/soas/test/ (We’re also storing the old test images so we can go back and forth between them Since the builds do contain their build date in their name, and we can’t predict ahead of time what the build date and time are, we don’t know the exact filename to pull.

So we’re basically looking for a shell script that will:

  1. Pull the latest iso and checksum from the SL servers
  2. Rename the checksum so it matches the datetime stamp of the iso (the checksum is currently called – rather unhelpfully – “CHECKSUM-i386″).
  3. Update the symlinks so that http://download.sugarlabs.org/soas/test/soas-i386-test-latest.iso and http://download.sugarlabs.org/soas/test/soas-i386-test-latest-checksum.sha point to the latest iso and checksum that were just downloaded.

This probably requires some sort of weird wildcard bash-fu that would take me multiple hours to inelegantly figure out, and someone else 5 minutes to write a one-liner to solve.

Can haz halp?

Know someone who'd appreciate this post?
  • Print
  • Facebook
  • Twitter
  • Google Bookmarks
  • email
  • Identi.ca
  1. 5 Responses to “Shell script ninja help needed: weekly test image downloading”

  2. I am very interested to do it.

    By Diaa on Jun 26, 2010

  3. Hey Mel,

    I would be extremely happy to help you create this script to do what you need. It’s that sort of thing I love to do.

    Ping me in IRC (herlo), or email me (you know it) when you get a chance and I’ll crank out a script for you.

    Cheers,

    Clint

    By Clint Savage on Jun 26, 2010

  4. Script at http://chris.tylers.info/files/soas-wget-test … I’ll post to the ticket when bugs.sugarlabs.org comes back up.

    By Chris Tyler on Jun 26, 2010

  5. In Ubuntu to handle the “downloading a whole iso daily is a lot of bandwidth” thing, we have zsync setup, so people wanting to go from one already-downloaded iso to the other just run a zsync command and it only has to download the diff.

    By Mackenzie on Jun 26, 2010

  1. 1 Trackback(s)

  2. Jun 27, 2010: [M]etabrain [E]ntry [L]og » Blog Archive » to-do braindump, Sunday morning

What do you think?