Adding wordpress.com blog as connected site in Google Buzz

Works like a charm.  If you’re not a wordpress.com site you can add the <meta> tag by going to the theme editor and inserting the tag in the header.php file (assuming that the theme is created properly).

Yay!

My first impression is that survey respondents don’t understand what the cloud is, but that shouldn’t have been a shock to me, most people don’t know what it is.  Admittedly this lack of understanding could be due to the way the survey was structured; but even after a couple of tweaks to the order of sections (I moved Software to the front of the survey and Platforms and Infrastructure to the end), respondents were still having a hard time understanding what was what.  I’ll most likely need to do some clean up on the survey, but I think for now I have a great understanding of what is going on in Higher Education in terms of the cloud.

In the survey I tried to define the three main components of the cloud: Software-as-a-Service, Platform-as-a-Service, and Infrastructure-as-a-Service.  After the survey was completed, I realized that I’ll need to do a better job of defining things for the audience members in my NERCOMP presentation.  Something that I though I could spend 10 minutes on will probably need a solid 20 minutes so that everyone can be on the same page.  I think diagrams and other visual aids might really help people understand what these different components are, and how they correspond to computing they are already using.

My second impression is that institutions are very comfortable with using Software-as-a-Service.  Below is a graph showing SaaS usage among respondents.  Facebook is of course the leader in the SaaS cloud race, with Twitter and Google Docs coming in right behind them.  What I think is the most interesting though is that overall, Google has the highest share of the marketplace.

As far as PaaS and IaaS, most institutions are not using these services.  I’m hesitant to show results from these sections of the survey since so many respondents confused software for platforms or infrastructure.  Once I clean things up, I’ll provide more information.  Suffice to say though, few if any institutions are using Infrastructures or Platforms in the cloud.  Those that are are using Amazon Web Services (for infrastructure) and Google Code (for platform).

More to come later.  And thank you to everyone that took the time to take my survey.  It has been eye opening and hopefully will make me a more informed speaker.

When I saw the post I printed it out (i know…the trees) and started marking it up.  This morning when I came back into the office I took a look at the paper and realized that most of the ideas that I liked were basically screaming “I’m a project!”  Some of the highlights from the post were:

• create a user-base for your data
• market to that user-base
• think about publishing your data at the beginning of the project
• consider the sustainability of publishing your data
• think about what you’re hoping to accomplish with your open data
• who are you targeting by opening up your data
• build your project based on what you want to accomplish and who you’re targeting

None of these points are really that new to me (or to anyone that works in systems).  You need to plan before starting on a project, and if you plan right you can have an awesome project.  I think planning is what makes data.gov.uk so much better than data.gov.  It was well thought out.  They considered user-bases in advance.  They incorporated RDF into the data catalogue (yes that’s the british spelling…we are talking about a british site afterall).  These little subtleties are what people are the most excited about.

At the same time I recognize that a lot of times we’re working with retrospective data, data that we thought no one would ever want to take a look at.  But that doesn’t mean you can’t make it useful now, and data.gov.uk proves that. Through great planning they created a very useful tool.

So my point is PLAN PLAN PLAN.  A well thought out plan can make a project succeed or fall flat.

As a companion to this presentation. I’m doing a survey, to see what kind of things institutions are using in the cloud. Its definitely not a comprehensive list of what’s in the cloud or even what’s possible in the cloud, but my presentation is only 40 minutes.

At any rate, if you work in academia and you’re interested in/using cloud computing, please please please take my survey.

http://www.surveymonkey.com/s/cloud-academia

Your help will be greatly appreciated.

First I always recommend keeping up the old installation while you’re migrating and not taking it down for at least a week or two (perhaps even longer if you’re paranoid).  That way if you need to, you can always go back.

Here are the steps that I followed:

1. I installed a blank MDID2 to make sure that I had everything I needed on the server to make MDID2 work.  When installing MySQL I made sure that it was the same version as the old server to ensure that MySQL weirdness didn’t interfere with the installation.  For instructions on installing MDID2, I used the MDID wiki’s instructions.  I actually ran the system and had our Visual Resources Curator test things out to see if it worked like he expected a fresh MDID would work (because lets face it, I don’t know nearly as much about the system as he does).
2. Once our Visual Resources Curator gave me the thumbs up I mounted, as a network drive, the directory on the old server that had the MDID2 files and images.  I then copied ALL of the files over to the new directory on the new server.  I then went home for the night…because that’s going to take a while.
3. The next morning go into the config.xml file (found at the root level of mdid2) and change the information in the file so that it’s settings match your new servers configuration.  Most likely all you’ll need to do is change the location of the image files or the username and password of the database, but you should check over the rest of the file just to be safe.
4. Create a mysqldump of the old database using: mysqldump -u mdid2 -p -q mdid2 > mdid2export.sql I then moved the dump over to the new server.  Keep in mind that your MySQL command may be different if:
   • you don’t use passwords
   • have a different MySQL user set to run mdid2
   • your mdid2 user doesn’t have privileges to do this type of command.
5. Once I got the dump onto the new server I opened the dump file in notepad and did a find replace.  In my case on the old server mdid was on the d drive and on the new server it was going to be on the c drive.  So I had notepad go through the file and find all instances of d:\\mdid2\\ and replace it with c:\\mdid2\\.  Again this may be different for you, depending on how your installation is set up.  Once this started I went to go get myself some tea, because this was going to take a bit (although not as long as moving all those files over).
6. Now that the database file has been “fixed” you’re ready to import the database.
   1. Go into MySQL’s commandline and drop the existing mdid2 database: drop database mdid2;
   2. Recreate the mdid2 database: create database mdid2; The reason you just destroyed and recreated the database was because the database has structure and data attached to it based on the work you did in step 1, you want to overwrite that.
   3. import the mysql database by going to Window’s commandline and doing: mysql -u mdid2 -p mdid2 < mdid2export.sql
7. This step is key to avoiding any indexing weirdness you may see.  Trust me, I did these steps multiple times before finally figuring this out (thanks to the mdid listserv).  You’ll need to go in and delete all instances of the _ftindex directory in the image directory.  Go through each and every directory in the mdid2/image directory to get rid of this folder.  If you don’t you’ll find some weirdness in the indexing of data.  Don’t worry this directory will be rebuilt once you start up mdid2.
8. Start up IIS and wait a bit for the full text index to finish up.  If you want, you can log in to your newly migrated MDID2 and click background transactions to see if the indexing process is still running (it should be the second thing in the list).
9. Last but certainly not least, have your Visual Resources Curator check the system over and give their stamp of approval.  Without that, you might as well be dead in the water.

Like I stated above, this is the process that I followed.  Everyone’s systems are set up differently, but hopefully this can act as a guide of sorts to whatever poor schmuck needs to do something similar (ie me…so I don’t forget like I always seem to).

So I wrote back to my dad:

They forgot to say that he created the first library school at Columbia, but forgot to ask the institution if that was alright.  He also had a desire to change the way the world spelled, so he consistently dropped unneeded letters from words (he was originally named Melville).  He also created the American Library Association and was subsequently kicked out for sexual harassment.  Finally he moved to upstate New York and created a colony.  He refused to allow in persons of color and jews.  One can assume from this that he was a bigot.

Basically Dewey was crazy, and would hate most of the librarians that are in the profession today.  So I suspect that’s why the occasion went largely unnoticed.

The first thing I did was start with an already existing EBS AMI. Right now there aren’t too many available but they are growing.  I chose an Ubuntu 9.04 image since I’m going to install a stack on it, and I’ve already done *most* of it before on Ubuntu.

A quick note: it is possible to create an EBS AMI from an S3 AMI, but I wasn’t able to figure that out (quickly).  Although I did find a script that will do this for you: http://gist.github.com/249915.

Next, I wanted to get the Ubuntu server prepared.  I needed to:

• install mysql — http://www.howtogeek.com/howto/ubuntu/install-mysql-server-5-on-ubuntu/
• install ruby on rails — http://www.hackido.com/2009/04/install-ruby-rails-on-ubuntu-904-jaunty.html (up to and including step 6)
• install zlib — http://www.techsww.com/tutorials/libraries/zlib/installation/installing_zlib_on_ubuntu_linux.php
• install blacklight — http://projectblacklight.org (includes solr and a number of gem dependencies)

I did all of this.  One thing I noticed, Blacklight requires me to use non-standard ports (ie not 22 and 80).  While the server was running, I went into the AWS console and opened ports for its associated security group.  The changes took effect immediately.  I definitely liked that.

Finally, I went in to the AWS Management Console and chose “Create Image (EBS AMI)”.

Then I filled in the information including a description of my image:

and voila! EBS backed AMI.  the one caveat is:

DON’T TERMINATE THE INSTANCE UNTIL THE NEW AMI HAS FINISHED BEING CREATED

because then you’ll loose all the work that you have just done.  doh!

So I’m still here thinking. I’ll probably be posting on cloud computing a bit more, since I will be presenting with Michael Klein from Oregon State University, on the topic.  So look forward to that (or don’t…).

and if you don’t have a google wave account, here is what it looks like.  What’s nice about it is that you can customize the look with different background and font colors.  Also pretty cool, I can edit the wave from within my blog post.  I have to be logged into my wave account (either in another browser window or the same browser window if i’m using tabs).

I can definitely see the usefulness of this.  I’ve seen some conferences try to figure out ways to aggregate information.  By adding the twitter bot and other bots, I may be able to use wave as a quick and easy way to aggregate information.

A few bummers:

1. default colors are less than ideal, but luckily you can change that. so for example right now my code to embed the wave reads: wave id =”your-wave-id” bgcolor=”#FAEBD7″ (enclosed within brackets)
2. people that don’t have a wave account can’t see the wave.  here is what they see: this is kind of a big let down.  it would be useful if they could at least view the wave…but unfortunately that’s not the case.

the library world will start to organize around co-operative cataloging vendors (and I’m going to put OCLC into the vendor pool like Ed pointed out others are starting to do). Essentially libraries will need to make a decision; do you like Ex Libris’ KnowledgeBase, OCLC’s KnowledgeBase, or some other vendor’s KnowledgeBase.

Yesterday III’s founder announced the beginnings of his cataloging co-operative SkyRiver. They claim to have a number of development partners in the project, including one ARL library (if you’re wondering who the possibilities are, Marshall Breeding has a list of ARL libraries on his website broken down by ILS type.  note that it may be possible that a non III catalog is a member, but given the way III works, I would argue that chance is slim).  So at anyrate……

Let the cataloging co-operative wars begin.