Following some of the suggestions in the Book “Release It”  (thanks to Demian Neidetcher for recommending the book in a blog post) I decided to implement an ops DB to track performance of specific components at work.  We store timing metrics in Log4J logs.   Since  log parsing was involved, and I wanted to get a prototype in place quickly, I reached for my favorite text parsing tool – perl.  After I got my log parsing regex’s working, it was time to store the data.  Finding hardware at work to set up another production DB can be a pain, so I thought I’d give Amazon SimpleDB a try.

With a bit of searching, I found a perl API provided by Amazon.   What was much harder to find were examples of how to use the API.   Amazon provides samples, but the samples don’t run out of the box  and are light on setup details.   After some effort, I got all the functions working.   I’ve documented examples below in hopes of helping out other perl hackers who need to use SimpleDB.  There are examples for creating, populating, and then cleaning up a SimpleDB datastore. A few notes before you look at the code:

— These examples work with the amazon-simpledb-2007-11-07-perl-library, which as far as I can tell is the most recent version of perl APIs provided by Amazon.  As of January 2013, this link worked: http://s3.amazonaws.com/awscode/amazon-simpledb/2007-11-07/perl/library/amazon-simpledb-2007-11-07-perl-library.zip

— The perl library was developed back in the early days of  Amazon SimpleDB.   Two methods, Query and QueryAttributes,  were available then but are now are not supported by SimpleDB.  Use the more flexible Select instead.

— All of these samples use the invokeXXXX subroutines provided in the perl API distribution.   I don’t show them below – you can just get them from the API.

Now, on with the code.

1. Create a domain

Creating a domain to store items is the first thing you need to do.

use Amazon::SimpleDB::Model::CreateDomainRequest;
my $request = Amazon::SimpleDB::Model::CreateDomainRequest->new(
{ DomainName => "$domain_name" } );

invokeCreateDomain( $service, $request );

2. List domains

use Amazon::SimpleDB::Model::ListDomainsRequest;

invokeListDomains( $service, $request );

3. Put an item with attributes in the store

Here’s where I found the perl API confusing.  You call “setAttribute” on an item with reference to an array of ReplaceableAttributes.  Since you’re passing an array, shouldn’t it be called setAttributes instead of setAttribute?

use Amazon::SimpleDB::Model::PutAttributesRequest;
use Amazon::SimpleDB::Model::ReplaceableAttribute;

my $first_item = "first_item";

my $attribute1 =
  Amazon::SimpleDB::Model::ReplaceableAttribute->new(
        {
            Name    => "flavor",
            Value   => "cherry",
            Replace => 0,
        }
  );

my $attribute2 =
  Amazon::SimpleDB::Model::ReplaceableAttribute->new(
        {
            Name    => "color",
            Value   => "red",
            Replace => 0,
        }
  );

$request =
  Amazon::SimpleDB::Model::PutAttributesRequest->new(
        {
            DomainName => "$domain_name",
            ItemName   => "$first_item",
        }
  );

my @attributes;

push @attributes, $attribute1;
push @attributes, $attribute2;

#IMPORTANT - pass the array reference, not the array!
$request->setAttribute( \@attributes );
invokePutAttributes( $service, $request );

4. Get the item you just stored

use Amazon::SimpleDB::Model::GetAttributesRequest;
$request =
  Amazon::SimpleDB::Model::GetAttributesRequest->new(
        {
           DomainName => "$domain_name",
           ItemName   => "$first_item",
        }
  );
invokeGetAttributes( $service, $request );

5. Use batch put  to store several items at once

Batch put is preferred for efficiency reasons.

use Amazon::SimpleDB::Model::BatchPutAttributesRequest;
use Amazon::SimpleDB::Model::ReplaceableItem;
use Amazon::SimpleDB::Model::ReplaceableAttribute;

my $attribute =
  Amazon::SimpleDB::Model::ReplaceableAttribute->new(
        {
            Name    => "flavor",
            Value   => "cherry",
            Replace => 0,
        }
  );

$attribute2 =
  Amazon::SimpleDB::Model::ReplaceableAttribute->new(
        {
            Name    => "flavor",
            Value   => "beef jerky",
            Replace => 0,
        }
  );

my $replaceableItem1 =
  Amazon::SimpleDB::Model::ReplaceableItem->new(
        {
           ItemName => "item_a_1"
        }
  );

my $replaceableItem2 =
  Amazon::SimpleDB::Model::ReplaceableItem->new(
        {
            ItemName => "item_a_2"
        }
  );

my @attributes1;
push @attributes1, $attribute;

#assign first set of attributes to first item
$replaceableItem1->setAttribute(@attributes1);

my @attributes2;
push @attributes2, $attribute2;

#assign second set of attributes to second item
$replaceableItem2->setAttribute(@attributes2);

#stuff both items in an array
my @items;
push @items, $replaceableItem1;
push @items, $replaceableItem2;

$request = Amazon::SimpleDB::Model::BatchPutAttributesRequest->new(
    {
       DomainName => "$domain_name"

    }
);

$request->setItem( \@items );

invokeBatchPutAttributes( $service, $request );

6.  Use the Select DSL to query items

As mentioned above,  Query and QueryAttributes have been dropped.  Select will do everything those did with a small SQL-like DSL.

use Amazon::SimpleDB::Model::SelectRequest;

my $query = "select * from $domain_name where flavor = 'cherry' limit 5";
$request = Amazon::SimpleDB::Model::SelectRequest->new(
    {
       SelectExpression => $query

         # NextToken => { FieldValue => undef, FieldType => "string"},
    }
);

invokeSelect( $service, $request );

7.  Delete attribute (or item)

Note: If you want to delete the item, don’t add any attributes to the request, just give the domain name and the item name.

use Amazon::SimpleDB::Model::DeleteAttributesRequest;
use Amazon::SimpleDB::Model::Attribute;

$request =
  Amazon::SimpleDB::Model::DeleteAttributesRequest->new(
        {
           DomainName => "$domain_name",
           ItemName   => "$first_item",
        }
  );

my $attribute_to_delete =
  Amazon::SimpleDB::Model::Attribute->new(
        {
           Name  => "color",
           Value => "red"
        }
  );

my @attributes_to_delete;
push @attributes_to_delete, $attribute_to_delete;

$request->setAttribute(@attributes_to_delete);
invokeDeleteAttributes( $service, $request );

8. Finally, delete the domain

use Amazon::SimpleDB::Model::DeleteDomainRequest;
$request = Amazon::SimpleDB::Model::DeleteDomainRequest->new(
           { DomainName => "$domain_name" }
);

invokeDeleteDomain($service, $request);

As it turns out, Amazon SimpleDB wasn’t the right place for an ops DB. I wanted to roll up the data for weekly and monthly views, and SimpleDB doesn’t provide aggregation functions like min, max, average, etc. You have to roll your own, and, more importantly you have to pull all the data you just stored back out of SimpleDB to do that.