Backing Up Web Data Using cURL
More and more of my important data is stored on the web: my bookmarks at del.icio.us, my citations at CiteULike, and my notes at Backpack and Writeboard. I love these services, but I am fully aware that they could disappear at any time, taking my precious data with them. Fortunately, they all make it easy for me to back up my data to the location of my choice (in this case the SIMS servers, which are themselves backed up regularly).
Backing up Backpack data:
#! /bin/sh
curl -s -H 'X-POST_DATA_FORMAT: xml'
-o "/home/ryanshaw/backup/backpack/$$.xml"
-d '<request><token>[My Backpack API token]</token></request>'
http://rybesh.backpackit.com/ws/account/export
Backing up CiteULike data:
#! /bin/sh URL="http://www.citeulike.org/bibtex/user" DIR="/home/ryanshaw/backup/citeulike" cd "$DIR/ryanshaw" mv -f *.xml backup/ curl -s "$URL/ryanshaw" | /home/ryanshaw/bin/bib2xml -s 2> /dev/null
Backing up del.icio.us data:
#! /bin/sh
curl -s --user rybesh:[my password]
-o "/home/ryanshaw/backup/delicious/$$.xml"
http://del.icio.us/api/posts/all
Backing up Writeboard data:
#! /bin/sh
url="http://123.writeboard.com"
dir="/home/ryanshaw/backup/writeboard"
cookies="$dir/cookies.txt"
# args: writeboard_id, password
export_writeboard () {
[ -d "$dir/$1" ] || mkdir "$dir/$1"
curl -s -L -c $cookies -o /dev/null -d "password=$2" "$url/$1/login"
curl -s -b $cookies -o "$dir/$1/$$.html" "$url/$1/v/export?format=html"
rm -f $cookies
}
export_writeboard [writeboard ID from URL] [writeboard password]
February 10th, 2009 at 3:31 pm
Hi there,
thanks for the curl script. I was just willing to write on myself, and was browsing for the delicious api, when I found your script. Works like a charm.