Getting Started with HCP REST API: Creating an Object (C/Curl)

Document created by Wayzen Lin on Jun 17, 2013Last modified by Michael Ratner on Oct 10, 2017
Version 9Show Document
  • View in full screen mode

Introduction

In this series of articles we'll walk you through some simple examples programs using the HCP REST API.  Our aim is to familiarize you with the basic methods of storing and retrieving data on HCP.  We want to quickly get you to the point where you can start writing useful applications.  Studying examples is usually the best way to start.

 

If you've worked in web development or have programmed applications using HTTP, you'll likely find the API pretty familiar.

 

Source code for this article is written in C/C++ using the open source library cURL to handle HTTP protocol transactions. Versions of this article using other languages will also be published and provided for your convenience.

 

Creating an Object

Let's get started.  Our first program uploads content from the file  hello.txt  to an HCP repository. We’ll give our new object the path hello/world.txt and publish it to the bucket namespace.

#include <stdio.h>

#include <stdlib.h>

#include <sys/stat.h>

#include <curl/curl.h>

 

#define AUTH_TOKEN "Authorization: HCP bXl1c2Vy:3f3c6784e97531774380db177774ac8d"

 

int main (int argc, char **argv)

{

        FILE *fh = NULL;                   /* file to upload */

        struct stat finfo;                 /* size of file to upload */

 

        CURL *curl = NULL;                 /* curl session handle */

        struct curl_slist *ch = NULL;      /* curl custom header list */

        CURLcode retval;                   /* curl return code */

 

        /* initialize curl */

 

        /* platform-specific setup */

        curl_global_init(CURL_GLOBAL_ALL);

 

        /* start session */

        curl = curl_easy_init();

        if(!curl) {

               fprintf(stderr, "Unable to initialize cURL HTTP library\n");

               goto cleanup;

        }

 

        /* prepare local file for upload */

 

        /* determine file size */

        if(stat(“hello.txt”, &finfo) < 0) {

               fprintf(stderr, “Unable to find file hello.txt\n”);

               goto cleanup;

        }

 

        /* open file handle */

        fh = fopen("hello.txt", "r");

        if(!fh) {

               fprintf(stderr, "Unable to open file hello.txt\n");

               goto cleanup;

        }

 

        /* build HTTP request */

 

        /* create an object using the PUT method, */

        curl_easy_setopt(curl, CURLOPT_PUT, 1L);

 

        /* addressed by this URL, */

        curl_easy_setopt(curl, CURLOPT_URL,

               "http://bucket.customer.hcp.foobar.com/rest/hello/world.txt");

 

        /* with content from the file referenced by this handle, */

        curl_easy_setopt(curl, CURLOPT_READDATA, fh);

 

        /* using curl's default file reader, */

        curl_easy_setopt(curl, CURLOPT_READFUNCTION, NULL);

 

        /* providing the file size as a hint, */

        curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, finfo.st_size);

 

        /* with the security token referenced in the Authorization header */

        curl_slist_append(ch, AUTH_TOKEN);

        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, ch);

 

        /* execute */

 

        /* run HTTP transaction as defined */

        retval = curl_easy_perform(curl);

        if(retval != CURLE_OK) {

               fprintf(stderr, "Error writing hello.txt to HCP!\n");

               goto cleanup;

        }

 

    cleanup:

 

        /* close local file handle */

        if(fh) {

               fclose(fh);

               fh = NULL;

        }

 

        /* free custom headers */

        if(ch) {

               curl_slist_free_all(ch);

               ch = NULL;

        }

 

        /* free curl session */

        if(curl) {

               curl_easy_cleanup(curl);

               curl = NULL;

        }

 

        /* free platform bindings */

        curl_global_cleanup();

        return retval;

}

 

When you run this program, the HTTP request from your machine should look something like this:

PUT /rest/hello/world.txt HTTP/1.1

Host: bucket.customer.hcp.foobar.com

Authorization: HCP bXl1c2Vy:3f3c6784e97531774380db177774ac8d

Content-Length: 12

 

Hello World

 

The PUT method is used to create new objects. You can instruct cURL to issue an HTTP PUT by calling curl_easy_setopt(3) with the following options:

/* Use this function to define the method for an HTTP transaction */

curl_easy_setopt(curl, CURLOPT_PUT, 1L);

 

HCP uses URLs to uniquely address objects. Let’s break down the different components of an HCP URL.

 

authenticated-put-url-5.png


The URL for an HTTP transaction is set by calling curl_easy_setopt(3) with the following options:

/* Use this function to specify the target URL */

curl_easy_setopt(curl, CURLOPT_URL, "http://bucket.customer.hcp.foobar.com/rest/hello/world.txt");

 

URLs specify the path for an object, as well as the namespace and tenant you are writing to. If any directories in the path do not already exist, HCP creates them.

 

Namespaces are a way of virtualizing the physical storage managed by HCP. Resources are fully isolated; objects in one namespace are not visible to any others. Applications sharing a repository are typically given their own namespace to use. This is analogous to the concept of buckets in the Amazon S3 ecosystem.

 

Namespaces have a lot of powerful features that allow you to manage and govern data based on policy, but for now it's enough to understand that:

  • Object names and directories are case sensitive
  • Names can include non-printing characters
  • Object names and directories cannot exceed a combined length of 4,095 bytes

 

Namespaces are owned and managed by administrative entities called tenants.  Each tenant manages one or more namespaces.  Customers sharing a repository are typically given their own tenant to manage, and create namespaces according to need or demand.


HCP uses the Host header to determine which namespace and tenant you are writing to.  cURL automatically populates the Host header, extracting the information from the URL you provide.

 

WARNING!

If you are not using cURL to handle HTTP for you, be sure to transmit a properly formatted Host header with your request.  HCP will reject your transaction otherwise.

 

WARNING!

If you are using IP addresses in your URL, be sure to transmit a properly formatted Host header – complete with namespace and tenant information – with your request.  HCP will reject your transaction otherwise.

 

Each namespace has a set of data access accounts associated with it.  Passwords are used to authenticate valid users.  Applications must transmit an authentication token for an active account in order to access data stored in a namespace.  Authentication tokens are defined as a base64-encoded account username concatenated with a hash of the account password, using a colon as a delimiter.

base64(acct-username):md5(acct-password)

 

Let's build an example authentication token, step by step.

acct username     testUser

acct password     testPassword

 

base64(testUser)  dGVzdFVzZXI=

md5(testPassword) fed3b61b26081849378080b34e693d2e

 

calculated token  dGVzdFVzZXI=:fed3b61b26081849378080b34e693d2e

 

If you are developing on a UNIX-based system, you can use the following shell command to generate token parameters via the command-line:

> echo -n testUser | base64

> echo -n testPassword | md5sum

 

Authentication tokens are passed as a part of the HTTP request using the Authentication header with the format:

Authentication: HCP bXl1c2Vy:3f3c6784e97531774380db177774ac8d"

 

You can instruct cURL to populate the Authentication header by calling curl_easy_setopt(3) with the following options:

/* pass pre-calculated authentication token as a list */

curl_slist_append(ch, AUTH_TOKEN);

curl_easy_setopt(curl, CURLOPT_HTTPHEADER, ch);

 

The actual data to be uploaded to HCP is attached to the body of PUT requests.  cURL provides built-in functions to read data from a file for upload.

/* Use this function to specify a FILE handle to read from */

curl_easy_setopt(curl, CURLOPT_READDATA, fh);

 

/* Use this function with NULL a argument to let curl fread(3)

* the FILE handle specified above */

curl_easy_setopt(curl, CURLOPT_READFUNCTION, NULL);

 

You can also use cURL to trigger custom read behaviors using callback functions. For more details please see the cURL API documentation.

 

HCP uses standard HTTP/1.1 response codes to communication transaction status.

Common return values for PUT requests include:

 

Status

Summary

Description

201

Created

Success

403

Forbidden

Protocol disabled, or insufficient permissions

409

Conflict

Object already exists

413

File Too Large

Insufficient capacity

 

cURL maps HTTP responses to cURL-specific error codes by default.  For instance, the response:

HTTP/1.1 201 Created

is parsed by cURL and presented as the return value:

CURLE_OK

to the function curl_easy_perform(3). More detailed information is often necessary to properly handle errors. You can use the cURL function curl_easy_getinfo(3) to directly retrieve HTTP response fields. Let's update our program to print out HTTP response codes.

long status = 0;

    ...

 

    /* run HTTP transaction as defined */

    retval = curl_easy_perform(curl);

 

    /* print HTTP response code to console */

    curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &status); 

    if(status) {

        fprintf(stdout, "Received %ld status code\n", status);

    }

 

HCP returns additional processing details in the header block.  Let's interpret the HTTP response to our program:

 

authenticated-put-response-5.png


The X-HCP-Hash header is a checksum for the uploaded object.  When a request arrives the repository calculates a checksum for the data received, which it returns as part of the HTTP response. The checksum can be used by applications to ensure that data uploaded by the PUT request wasn't corrupted in-transit.

 

Conclusion

You should now have a basic understanding of how to create objects in HCP using the HTTP/REST interface.  You can find more detail on the API in the Using a Namespace guide found in the product documentation.

 

Happy coding!

Attachments

    Outcomes