Using Google Drive API with C# – Part 2

Welcome to Part 2 which covers the authorization process. If you have not yet set up your Google API access, please read part 1 first.

OpenAuth

The OpenAuth initially seems pretty complicated, but once you get your head around it, it’s not that scary, honest!

If you followed the steps in Part 1 you should now have a Client ID and Client Secret, which are the ‘username’ and ‘password’. However, these by themselves are not going to get you access directly.

Hotel OpenAuth

You can think of OpenAuth of being a bit like a hotel, where the room is your Google Drive. To get access you need to check in at the hotel and obtain a key.

When you arrive at reception, they check your identity, and once they know who you are, they issue a card key and a PIN number to renew it. This hotel uses electronic card keys and for security reasons they stop working after an hour.

When the card stops working you have to get it re-enabled. There is a machine in the lobby where you can insert your card, enter the PIN and get the card renewed, so you don’t have to go back to the reception desk and ask for access again.

Back To OpenAuth

In OpenAuth ‘Reception’ is the authentication request that appears in the browser you get when you first attempt to use a Client ID and Client Secret. This happens when you call GoogleWebAuthorizationBroker.AuthorizeAsync the first time.

This allows the user to validate the access being requested from your application. If the access is approved, the client receives a TokenResponse object.

The OpenAuth ‘key card’ is called an AccessToken, and will work for an hour after being issued. It’s just a string property in the TokenResponse. This is what is used when you try to access files or resources on the API.

When the AccessToken expires you need to request a new one, and the ‘PIN number’ is a RefreshToken (another property in TokenResponse) which also got issued when the service validated you. You can save the refresh token and re-use it as many times as you need. It won’t work without the matching Client ID and Client Secret, but you should still keep it confidential.

With the .NET API this renewal process is automatic – you don’t need to request a new access key if you’ve provided a RefreshToken. If the access is revoked by the Drive’s owner, the RefreshToken will stop working, so you need to handle this situation when you attempt to gain access.

Token Storage

The first time you make a call to AuthorizeAsync will result in the web authorization screen popping up, but in subsequent requests this doesn’t happen, even if you restarted the application. How does this happen?

The Google .NET client API stores these access requests using an interface called IDataStore. This is an optional parameter in the AuthorizeAsync method, and if you didn’t provide one, a default FileDataStore (on Windows) would have been used. This stores the TokenResponse in a file in a folder [userfolders]\[yourname]\AppData\Roaming\Drive.Auth.Store

When you call AuthorizeAsync a second time, the OpenAuth API uses the key provided to see if there is already a TokenResponse available in the store.

Key, what key? The key is the third parameter of the AuthorizeAsync method, which in most code samples is just “user”.

   1: var credential = GoogleWebAuthorizationBroker.AuthorizeAsync(

   2:                 secrets,

   3:                 new string[] {DriveService.Scope.Drive}, 

   4:                 "user", 

   5:                 CancellationToken.None).Result;

It follows that if you run your drive API application on a different PC, or logged in as a different user, the folder is different and the stored TokenResponse isn’t accessible, so the user will get prompted to authorise again.

Creating Your Own IDataStore

Since Google uses an interface, you can create your own version of the IDataStore. For my application, I would only be using a single client ID and secret for the application, but I wanted it to work on the live server without popping up a web browser.

I’d already obtained a TokenResponse by calling the method without a store, and authorised the application in the browser. This generated the TokenResponse in the file system as I just described.

I copied the value of just the RefreshToken, and created a MemoryDataStore that stores the TokenResponses in memory, along with a key value to select them. Here’s the sequence of events:

  1. When my application starts and calls AuthorizeAsync the first time, I pass in MemoryDataStore.
  2. The Google API then calls the .GetAsync<T> method in my class, so I hand back a TokenResponse object where I’ve set only the ResponseToken property.
  3. This prompts the Google OAuth API to go and fetch an AccessToken (no user interaction required) that you can use to access the drive.
  4. Then the API calls StoreAsync<T> with the resulting response. I then replace the original token I created with the fully populated one.
  5. This means the API won’t keep making requests for AccessTokens for the next hour, as the next call to GetAsync<T> will return the last key (just like the FileStore does).

Note that the ResponseToken we get back has an ExpiresInSeconds value and an Issued date. The OAuth system has auto-renewal (although I’ve not confirmed this yet) so when your AccessToken expires, it gets a new one without you needing to do this.

My code for the MemoryDataStore is as follows:

   1: using Google.Apis.Auth.OAuth2.Responses;

   2: using Google.Apis.Util.Store;

   3: using System;

   4: using System.Collections.Generic;

   5: using System.Linq;

   6: using System.Text;

   7: using System.Threading.Tasks;

   8:  

   9: namespace Anvil.Services.FileStorageService.GoogleDrive

  10: {

  11:     /// <summary>

  12:     /// Handles internal token storage, bypassing filesystem

  13:     /// </summary>

  14:     internal class MemoryDataStore : IDataStore

  15:     {

  16:         private Dictionary<string, TokenResponse> _store;

  17:  

  18:         public MemoryDataStore()

  19:         {

  20:             _store = new Dictionary<string, TokenResponse>();

  21:         }

  22:  

  23:         public MemoryDataStore(string key, string refreshToken)

  24:         {

  25:             if (string.IsNullOrEmpty(key))

  26:                 throw new ArgumentNullException("key");

  27:             if (string.IsNullOrEmpty(refreshToken))

  28:                 throw new ArgumentNullException("refreshToken");

  29:  

  30:             _store = new Dictionary<string, TokenResponse>();

  31:  

  32:             // add new entry

  33:             StoreAsync<TokenResponse>(key,

  34:                 new TokenResponse() { RefreshToken = refreshToken, TokenType = "Bearer" }).Wait();

  35:         }

  36:  

  37:         /// <summary>

  38:         /// Remove all items

  39:         /// </summary>

  40:         /// <returns></returns>

  41:         public async Task ClearAsync()

  42:         {

  43:             await Task.Run(() =>

  44:             {

  45:                 _store.Clear();

  46:             });

  47:         }

  48:  

  49:         /// <summary>

  50:         /// Remove single entry

  51:         /// </summary>

  52:         /// <typeparam name="T"></typeparam>

  53:         /// <param name="key"></param>

  54:         /// <returns></returns>

  55:         public async Task DeleteAsync<T>(string key)

  56:         {

  57:             await Task.Run(() =>

  58:             {

  59:                 // check type

  60:                 AssertCorrectType<T>();

  61:  

  62:                 if (_store.ContainsKey(key))

  63:                     _store.Remove(key);

  64:             });

  65:         }

  66:  

  67:         /// <summary>

  68:         /// Obtain object

  69:         /// </summary>

  70:         /// <typeparam name="T"></typeparam>

  71:         /// <param name="key"></param>

  72:         /// <returns></returns>

  73:         public async Task<T> GetAsync<T>(string key)

  74:         {

  75:             // check type

  76:             AssertCorrectType<T>();

  77:  

  78:             if (_store.ContainsKey(key))

  79:                 return await Task.Run(() => { return (T)(object)_store[key]; });

  80:  

  81:             // key not found

  82:             return default(T);

  83:         }

  84:  

  85:         /// <summary>

  86:         /// Add/update value for key/value

  87:         /// </summary>

  88:         /// <typeparam name="T"></typeparam>

  89:         /// <param name="key"></param>

  90:         /// <param name="value"></param>

  91:         /// <returns></returns>

  92:         public Task StoreAsync<T>(string key, T value)

  93:         {

  94:             return Task.Run(() =>

  95:             {

  96:                 if (_store.ContainsKey(key))

  97:                     _store[key] = (TokenResponse)(object)value;

  98:                 else

  99:                     _store.Add(key, (TokenResponse)(object)value);

 100:             });

 101:         }

 102:  

 103:         /// <summary>

 104:         /// Validate we can store this type

 105:         /// </summary>

 106:         /// <typeparam name="T"></typeparam>

 107:         private void AssertCorrectType<T>()

 108:         {

 109:             if (typeof(T) != typeof(TokenResponse))

 110:                 throw new NotImplementedException(typeof(T).ToString());

 111:         }

 112:     }

 113: }

This sample uses the following Nuget package versions:

   1: <package id="Google.Apis" version="1.8.2" targetFramework="net45" />

   2: <package id="Google.Apis.Auth" version="1.8.2" targetFramework="net45" />

   3: <package id="Google.Apis.Core" version="1.8.2" targetFramework="net45" />

   4: <package id="Google.Apis.Drive.v2" version="1.8.1.1270" targetFramework="net45" />

   5: <package id="log4net" version="2.0.3" targetFramework="net45" />

   6: <package id="Microsoft.Bcl" version="1.1.9" targetFramework="net45" />

   7: <package id="Microsoft.Bcl.Async" version="1.0.168" targetFramework="net45" />

   8: <package id="Microsoft.Bcl.Build" version="1.0.14" targetFramework="net45" />

   9: <package id="Microsoft.Net.Http" version="2.2.22" targetFramework="net45" />

  10: <package id="Newtonsoft.Json" version="6.0.3" targetFramework="net45" />

  11: <package id="Zlib.Portable" version="1.9.2" targetFramework="net45" />

Using Google Drive API with C# – Part 1

We had a requirement to store a large volume of user files (for call recordings) as part of a new service. Initially it would be a few gigabytes of MP3 files, but if successful would possibly be into the terabyte range. Although our main database server has some space available, we didn’t want to store these on the database.

Storing them as files on the server was an option, but it would then mean we had to put in place a backup strategy. We would also need to spend a lot of money on new disks, or new NAS devices, etc. It started to look complicated and expensive.

It was then the little “cloud” lightbulb lit up, and we thought about storing the files on a cloud service instead. I have stuff on Dropbox, LiveDrive SkyDrive OneDrive and a Google Drive. However the recent price drop on Google Drive meant this was the clear favourite for storing our files. At $120 per year for 1TB of space that’s a no-brainer.

Google API

To do this we’d need to have the server listing, reading and writing files directly to the Google Drive. To do this we needed to use the Google Drive API.

I decided to write this series of articles because I found a lot of the help and examples on the web were confusing and in many cases out-of-date: Google has refactored a lot of the .NET client API and a lot of online sample code (including some in the API documentation) is for older versions and all the namespaces, classes and methods have changed.

API Access

To use the Google APIs you need a Google account, a Google drive (which is created by default for each user, and has 30GB of free storage), and API access.

Since you can get a free account with 30GB you can have one account for development and testing, and kept separate from the live account. You may want to use different browsers to set up the live and development/testing accounts. Google is very slick at auto-logging you in and then joining together multiple Google identities. For regular users this is great, but when trying to keep the different environments apart it’s a problem.

User or Service Account?

When you use the Google Drive on the web you’re using a normal user account. However, you may spot that there is also a service account option.

It seems logical that you might want to use this for a back-end service, but I’d recommend against using this for two reasons:

Creating a Project

I’ll assume you’ve already got a Google Account: if not you should set one up. I created a new account with it’s own empty drive so that I could use this in my unit test code.

Before you can start to write code you need to configure your account for API access. You need to create a project in the developer’s console. Click create project and give it name and ID. Click the project link once it’s created. You should see a welcome screen.

APIs

On the menu on the left we need to select APIs & Auth – this lets you determine which APIs your project is allowed to use.

A number of these will have been preselected, but you can ignore these if you wish. Scroll down to Drive API and Drive SDK. The library will be using the API but it seems to also need the SDK (enlighten me please if this is not the case!) so select both. As I understand it, the SDK is needed if you’re going to create online apps (like Docs or Spreadsheet), rather than accessing a drive itself. The usual legal popups will need to be agreed to.

The two entries will be at the top of the page, with a green ON button, and a configuration icon next to each

image

Configuration is a bit.. broken at present. Clicking either goes to configuration for the SDK on an older page layout. I don’t know if the SDK configuration is required or not. You could try accessing without it.

Credentials

The next step is to set up credentials. These are the identity of your login and there are different clientIDs based on the type of client you want to run.

By default you will have Client IDs set up for Compute Engine and Client ID for web application. To access drive from non-web code you need a Native Application, so click Create New Client ID. Select Installed application type and for C# and other .NET apps, select Other.

When this has completed you’ll have a new ClientID and a ClientSecret. Think of this as a username and password that you use to access the drive API. You should treat the Client Secret in the same way as a password and not disclose it. You might also want to store it in encrypted form in your application.

next: Part 2 – Authorising Drive API

Excel Art – The New ASCII Art?

Reading my daily CodeProject newsletter today, I came across this article about a Japanese artist who uses the graphic editing tools in Excel to create art.

Like many commenters, I disappointed that he hadn’t actually used Excel cells to create the image in the style of Pointillism.

If you know C# this is not a problem, and a few minutes later we have this code:

            // gembox spreadsheet key set
            GemBox.Spreadsheet.SpreadsheetInfo.SetLicense("[redacted]");

            // create the Excel output file
            var xl = new GemBox.Spreadsheet.ExcelFile();
            var ws = xl.Worksheets.Add("IBMPC");

            const string imageResource= "ExcelPaint.IBM PC 5150.jpg";

            var thisEXE = System.Reflection.Assembly.GetExecutingAssembly();
            using (var resource = thisEXE.GetManifestResourceStream(imageResource)) 
            {
                var img = new Bitmap(resource);
                for (int x = 0; x < img.Width; x++)
                {
                    for (int y = 0; y < img.Height; y++)
                    {
                        // get pixel colour from image
                        Color pixel = img.GetPixel(x, y);

                        // write to Excel cell
                        ws.Cells[y, x].Style.FillPattern.PatternForegroundColor= pixel;
                        ws.Cells[y, x].Style.FillPattern.PatternStyle = GemBox.Spreadsheet.FillPatternStyle.Solid;
                    }
                }
            }

            xl.SaveXlsx("C:\\IBMPC5050.xlsx");

I use the excellent GemBox SpreadSheet library for my Excel interaction. The code should not really need any explanation: read a pixel, write a “pixel”.

The resulting output is an Excel Spreadsheet, as shown below. Initially the aspect-ratio of the image was distorted as the cells were not square. I manually resized the column widths to 2, and it looks correct.

image

You can download the resulting output yourself here. I have to admit this is not going to win awards for image compression techniques: the original JPG was only 43KB and the Excel sheet was 1291KB. But hey, it’s 100% cooler.

For reference this is the original image:

In case you’re wondering – why an old PC as a test picture? This was the first suitable image I found, it’s one of my collection of old computers. It’s also a reference to the ASCII art of days gone by.