Canvas Data (API) 401:Unauthorized

[jack0x539]

I'm attempting to access Canvas Data, using the API, following the guide set out by  @James  -Canvas Data API Authentication, but I'm running into a 401. Hopefully I'm just missing something simple, if anyone can see it I'd be grateful.

I'm using .NET, but the code shouldn't differ much for any other language. I've pasted my code below in case someone can spot anything untoward, or missing.

Please note that although I can't rule out there being something amiss with my GenerateHMACSignature function, it does produce the desired HMAC signature when using the example parameters given in James' article.

Generating HMAC token:

static string GenerateHMACSignature(string secret, string url, DateTime timestamp)
{
   var uri = new Uri(url);
   string query = "";
   if (uri.Query.Length > 1)
   {
      var queryParams = uri.Query.Substring(1).Split('&').OrderBy(q => q);
      query = Combine(queryParams, "&");
   }
   var parts = $"GET\n{uri.Host}\n\n\n{uri.AbsolutePath}\n{query}\n{timestamp:r}\n{secret}";

   using (HMACSHA256 hmac = new HMACSHA256(Encoding.Default.GetBytes(secret)))
   {
      var hash = hmac.ComputeHash(Encoding.Default.GetBytes(parts));
      return Convert.ToBase64String(hash);
   }  
}

Making the request:

var timestamp = DateTime.Now;
var url = "https://portal.inshosteddata.com/api/account/self/dump?limit=100&after=45";

var signature = GenerateHMACSignature(apiSecret, url, timestamp);
var request = WebRequest.CreateHttp(url);
request.Headers["Authorization"] = $"HMACAuth {apiKey}:{signature}";
request.Date = timestamp;

[James]

jack0x539,

I'm not a .NET programmer, so this may be headed down the wrong track, but the thing that jumps out at me is the timestamp.

What format and timezone does Datetime.Now() come out in? When I look at DateTime.Now Property (System)  it appears that you might need to add a ToString() method.

I found these two suggestions on StackOverflow.

DateTime.UtcNow.ToString("o")

or

DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ")

[jack0x539]

james.vlisides,

Thanks for taking the time to read through this.

I'm using the string format specifier "r", which produces Tue, 23 May 2017 15:31:10 GMT, bringing my total string to be hashed to be the following:

GET\nportal.inshosteddata.com\n\n\n/api/account/self/dump\nafter=45&limit=100\nTue, 23 May 2017 15:31:10 GMT\nMY_SECRET_REMOVED_BEFORE_POSTING

This is totally different to the "o" or "yyyy-MM-ddTHH:mm:ssZ" that you're suggesting, which would yield 2017-05-23T16:31:10.0000000+01:00, but I thought I was following your tutorial's advice when using the former format ?

[edit:typo correction]

[James]

Sorry, I was confusing the Canvas REST API with the Canvas Data API.

I still think the timestamp is your issue. You are formatting it in the string that gets parsed by the HMAC, but you're not formatting it in the request.date header. I didn't even see the 'r' thing in the signature generation, I was just looking at the bottom part. They need to match -- exactly.

I would let var timestamp = DateTime.Now.ToString('r') at the beginning and just work with it as a string (if that's not valid C, please fix it)

[jack0x539]

Problem there is that the Date header for a .NET WebRequest is strongly typed, so I can't assign a string to it. To try and rule out builtin formatting issues, I just ran through my application while Fiddler was active, and here're the headers set:

GET https://portal.inshosteddata.com/api/account/self/dump?limit=100&after=45 HTTP/1.1
Authorization: HMACAuth API_KEY_REMOVED:cWL5m2WfyEFfabZ7IF5ffMlJZG36u3bJ02xIgE5K6og=
Date: Tue, 23 May 2017 15:05:51 GMT
Host: portal.inshosteddata.com
Connection: Keep-Alive

This looks to me like the date is coming out in the format that I want.

[James]

The timestamp you gave in the last response is earlier than the one you gave in the first one

Now: Tue, 23 May 2017 15:05:51 GMT

Before: Tue, 23 May 2017 15:31:10 GMT

I obviously don't know what I'm talking about with .NET, but one of the restrictions was that the time, whatever you put in, must be within 15 minutes of the server time and you said you "just" did it, so one of the times is wonky. It's probably something else, but I just wanted to throw that out there to make sure it's generating the correct time.

The other thing I would do is eliminate the query parameters and get it working with just a basic call first. I don't think that's the issue you said you were able to generate the right code with my example data and it's an unauthorized 401 error not an invalid parameter error, but you try to get the simple to work first. Still, if the query part was somehow off, it would mess up the HMAC signature and that could lead to an unauthorized error. Anyway, start with the simple to make sure to rule things out.

When I was testing this out, I went into a REST client (I use Advanced Rest Client for Chrome but a lot of people like Postman) and test it out manually. Log all of the information that you're getting and then put it in and make the call. You've got 15 minutes so it's not a big rush. If it works there, then it's something in your implementation. You didn't show the line that actually makes the call to the API, so I'm not sure what happens there (I probably wouldn't understand it anyway).

You can double check your API secret and key and make sure that you got them transferred correctly.

You might give the Canvas Data command line tool a try and see if you can get it to work. After all the work I did figuring out how to make the HMAC signature and do the calls, I ended up just using it.

If none of that helps, then I hope someone else steps as I'm running out of ideas.

[jack0x539 · Accepted solution]

James, 

The REST client helped me, thanks, it did come down to the time, as you said, but it's me misunderstanding the date formatting in .NET. For anyone else who cares:

DateTime.ToString("r") was returning Wed, 24 May 2017 14:57:06 GMT, which mislead me, as the time is not GMT, the time was BST (British Summer Time), yet the time zone, was still GMT.

I ended up using, DateTime.ToUniversalTime().ToString("r"), which returns Wed, 24 May 2017 13:57:06 GMT, and then it works perfectly.

Thanks for forcing me to look at the date part in detail, I was getting a little wound up with it!

[James]

I'm glad you were able to figure it out and that it was something I could help with because I was definitely not understanding the .NET stuff.

[ilahat]

Hey guys, I am using the same code as above but still getting 401. I tried all kind of different dates and formats, nothing seem to work. 

static string GenerateHMACSignature(string secret, string url, string timestamp)
{
var uri = new Uri(url);
string query = "limit=100&after=45";
// if (uri.Query.Length > 1)
// {
// var queryParams = uri.Query.Substring(1).Split('&').OrderBy(q => q);
// query = Combine(queryParams, "&");
//}
var parts = "GET\n" + uri.Host + "\n\n\n" + uri.AbsolutePath + "\n" + query + "\n" + timestamp + "\n" + secret;

using (HMACSHA256 hmac = new HMACSHA256(Encoding.Default.GetBytes(secret)))
{
var hash = hmac.ComputeHash(Encoding.Default.GetBytes(parts));
return Convert.ToBase64String(hash);
}
}

public void test()
{
DateTime timestamp = DateTime.Now;
string timestamp_st = DateTime.Now.ToUniversalTime().ToString("r");
var url = "https://portal.inshosteddata.com/api/account/self/dump";
string apiKey = "fsdfsd";
string secret = "fsdfsds";
var signature = GenerateHMACSignature(secret, url, timestamp_st);
var request = WebRequest.CreateHttp(url);
request.Headers["Authorization"] = "HMACAuth "+ apiKey+":"+signature;
request.Date = timestamp;
WebResponse response = request.GetResponse();
Stream dataStream = response.GetResponseStream();
string stat = (((HttpWebResponse)response).StatusDescription);
}

I really dont understand why CANVAS cant use normal authentication methods.

[James]

 @ilahat  

I don't speak .NET, but your code isn't the same as the code above.

What it looks like you're doing is forcing the query string to be "limit=100&after=45". All that code you commented out was designed to fulfill requirement #6 from the existing documentation section in Canvas Data API Authentication that says the query parameters have to be in alphabetical order. That would make after come before limit.

Try using "after=45&limit=100", or better yet, don't put hardcode the querystring into the function that computes the signature. It doesn't belong there as it makes it useless for other calls.

[ilahat]

Thanks that was the problem. I really dont get why making a REST call is so complicated you cant do it without writing code. At the end of the its just to download the files that are already on s3, it would been much simpler and more secure to give sftp access with a key file or access to the s3 bucket.