Rails 4 Application Development HOTSHOT (2014)

Chapter 8. API Only Application - Backend for a Mobile App

Android and iOS have taken smartphones to a new level of sophistication, and their respective application ecosystems have created a huge movement among developers. In order to do so, a lot of these applications need background processing, data storage, and data manipulation, along with authentication and authorization. In such cases, we need an API only application, where the application handles the processing load and returns data to a mobile client using an API.

Mission briefing

In an API only application, only models and controllers exist. This kind of application provides an interface to the data, apart from a full-blown application with business logic in the backend. The application can serve as a backend to a mobile application. The application only behaves like an API, with no UI layer present for the data. We will create an application to make notes and send data to an application using a REST client. As we create an API, we will also see how to create an OAuth2 provider to authorize applications with it.

The following screenshot shows how our application's OAuth provider screen will look at the end of our project:

Mission briefing

Why is it awesome?

Rails has a subproject called Rails API that takes a leaner approach to the creation of these applications. It strips off a lot of middleware components and the view layer out of the application. We can conditionally add some middleware and controller modules wherever required. We will also convert our application into the OAuth provider so that the authentication on our mobile applications happens via OAuth. We will use the doorkeeper gem to create an OAuth application ID and OAuth secret, along with a callback URL.

At the end of this project, we will be able to create an API only application with OAuth end points to authorize applications for the data API.

Your Hotshot objectives

While building this application, we will have to go through the following tasks:

·        Creating, editing, and deleting notes

·        Arranging notes category wise

·        Sending join and association data via JSON

·        Creating an OAuth2 provider

·        Generating API keys

·        Securing the application

Mission checklist

We need to install the following on the system and sign up for the API keys before we start with our mission:

·        Ruby 1.9.3 / Ruby 2.0.0

·        Rails 4.0.0

·        MySQL

·        Rails API

·        Devise

·        Git

·        Doorkeeper

·        jQuery

Creating, editing, and deleting notes

Our first task involves certain tasks that have already been done in the previous projects, but now we are going to do it in a slightly different way. As the main aim of our application is to be leaner and faster than a normal application, we will add some rack, action controller, and action view modules only when required. Some middleware stack (rack modules) and view-related stack (assets and views) components are stripped off. Please be sure to go through the Readme section of Rails API in detail (https://github.com/rails-api/rails-api/blob/master/README.md). It contains a list of modules that are included in a default Rails API application and what they are used for. It also has a list of modules that can be included in order to extend the default stack as and when required.

Engage thrusters

We will start by installing Rails API and generating our skeleton application by performing the following steps:

1.    We will install Rails API first using the following command:

2.  $ gem install rails-api

3.    Once the gem is installed, we will generate a blank application using the gem. We will use MySQL to develop our application as follows:

4.    rails-api new notely -d mysql

5.      create 

6.      create  README.rdoc

7.      create  Rakefile

8.      create  config.ru

9.      create  .gitignore

10.    create  Gemfile

11.    create  app

12.    create  app/controllers/application_controller.rb

13.    create  app/assets/images/.keep

14.    create  app/mailers/.keep

15.    create  app/models/.keep

16.    create  app/controllers/concerns/.keep

17.    create  app/models/concerns/.keep

18.    create  bin

19.    create  bin/bundle

20.    create  bin/rails

21.    create  bin/rake

22.    create  config

23.    create  config/routes.rb

24.    create  config/application.rb

25.    create  config/environment.rb

26.    create  config/environments

27.    create  config/environments/development.rb

28.    create  config/environments/production.rb

29.    create  config/environments/test.rb

30.    create  config/initializers

31.    create  config/initializers/secret_token.rb

32.    create  config/initializers/wrap_parameters.rb

33.    create  config/locales

34.    create  config/locales/en.yml

35.    create  config/boot.rb

36.    create  config/database.yml

37.    create  db

38.    create  db/seeds.rb

39.    create  lib

40.    create  lib/tasks

41.    create  lib/tasks/.keep

42.    create  lib/assets

43.    create  lib/assets/.keep

44.    create  log

45.    create  log/.keep

46.    create  public

47.    create  public/404.html

48.    create  public/422.html

49.    create  public/500.html

50.    create  public/favicon.ico

51.    create  public/robots.txt

52.    create  test/fixtures

53.    create  test/fixtures/.keep

54.    create  test/controllers

55.    create  test/controllers/.keep

56.    create  test/mailers

57.    create  test/mailers/.keep

58.    create  test/models

59.    create  test/models/.keep

60.    create  test/helpers

61.    create  test/helpers/.keep

62.    create  test/integration

63.    create  test/integration/.keep

64.    create  test/test_helper.rb

65.      run  bundle install

66.Fetching gem metadata from https://rubygems.org/

67.Fetching gem metadata from https://rubygems.org/..

68.Resolving dependencies...

69.Using rake (10.1.0)

70.Using i18n (0.6.9)

71.Using minitest (4.7.5)

72.Using multi_json (1.8.2)

73.Using atomic (1.1.14)

74.Using thread_safe (0.1.3)

75.Using tzinfo (0.3.38)

76.Using activesupport (4.0.1)

77.Using builder (3.1.4)

78.Using erubis (2.7.0)

79.Using rack (1.5.2)

80.Using rack-test (0.6.2)

81.Using actionpack (4.0.1)

82.Using mime-types (1.25.1)

83.Using polyglot (0.3.3)

84.Using treetop (1.4.15)

85.Using mail (2.5.4)

86.Using actionmailer (4.0.1)

87.Using activemodel (4.0.1)

88.Using activerecord-deprecated_finders (1.0.3)

89.Using arel (4.0.1)

90.Using activerecord (4.0.1)

91.Using bundler (1.3.5)

92.Using hike (1.2.3)

93.Using mysql2 (0.3.14)

94.Using thor (0.18.1)

95.Using railties (4.0.1)

96.Using tilt (1.4.1)

97.Using sprockets (2.10.1)

98.Using sprockets-rails (2.0.1)

99.Using rails (4.0.1)

100.   Using rails-api (0.1.0)

101.   Your bundle is complete!

102.   Use `bundle show [gemname]` to see where a bundled gem is installed.

103.     Rails API is under constant development, and its compatibility with Rails 4 is being improved day by day. In order to avoid some bugs and pitfalls, we will bundle Rails API from the master as follows:

104.   Gemfile


106.   gem 'rails', '4.0.1'

107.   gem 'rails-api', git: 'https://github.com/rails-api/rails-api.git', branch: 'master'

108.     Set up database.yml according to your local machine credentials.

109.     Now that our Rails API base project is set up, we will generate a model and a controller to create our notes as follows:

110.     notely$rails g scaffold note title:string body:string

111.       invoke  active_record

112.       create    db/migrate/20131211143114_create_notes.rb

113.       create    app/models/note.rb

114.       invoke    test_unit

115.       create      test/models/note_test.rb

116.       create      test/fixtures/notes.yml

117.       invoke  api_resource_route

118.       route    resources :notes, except: [:new, :edit]

119.       invoke  scaffold_controller

120.       create    app/controllers/notes_controller.rb

121.       invoke    test_unit

122.       create      test/controllers/notes_controller_test.rb

123.     We will add a very basic version for our API. In order to do this, we will create a namespace in routes:

124.   config/routes.rb


126.     Notely::Application.routes.draw do

127.     namespace :api do

128.       namespace :v1 do

129.         resources :notes

130.       end

131.     end



134.   end

135.     This will give an error because we need to create the same namespace in the controllers folder and move our notes_controller.rb file to api/v1 as follows:

136.   notely/app/controllers$mkdir api

137.   notely/app/controllers$ cd api

138.   notely/app/controllers/api$mkdir v1

139.   notely/app/controllers/$cd ..

140.   notely/app/controllers/$mv notes_controller.rb api/v1

141.     Our application_controller.rb file extends from ActionController::API as follows:

142.   app/controllers/application_controller.rb

143.   class ApplicationController < ActionController::API


145.   end

146.     Now that we have created a namespace, we also need to convert our controller to work with this namespace. We will also refactor the code to use before_action in order to set the note id for multiple actions as follows:

147.   app/controllers/api/v1/notes_controller.rb

148.   #namespace for api

149.   module Api

150.     module V1

151.       class NotesController < ApplicationController



154.       def index

155.         @notes = Note.all

156.         render json: @notes

157.       end


159.       def new

160.         @part = Part.new

161.       end


163.       def show


165.        @note = Note.find(params[:id])


167.         render json: @note

168.       end


170.       def create

171.         @note = Note.new(params[:id])

172.         if @note.save

173.           render json: @note, status: :created, location: @note

174.         else

175.         render json: @note.errors, status: :unprocessable_entity

176.       end

177.     end


179.       def update


181.         @note = Note.find(params[:id])


183.         if @note.update(params[:id])

184.           head :no_content

185.         else

186.           render json: @note.errors, status: :unprocessable_entity

187.         end

188.       end


190.       def destroy


192.         @note.destroy

193.         head :no_content

194.       end




198.     end

199.    end

200.   end

201.     We have to make Rails API and Rails 4 compatible with strong parameters because we need to post data to the API and pass the data as parameters in our client. We will first create an initializer to load a StrongParameters module along with theActionController::API module as follows:

202.   config/initializers/strong_parameters.rb

203.   ActionController::API.send :include, ActionController::StrongParameters

204.     We also need to modify our controller to the Rails 4 format by passing params through a private method. The default controller generated by Rails API is not in accordance with the Rails 4 format to accept the parameters:

205.   app/controllers/api/v1/notes_controller.rb


207.   private

208.     def set_note

209.       @note = Note.find(params[:id])

210.     end


212.     # Never trust parameters from the scary internet, only allow the white list through.

213.     def note_params

214.       params.require(:note).permit(:title, :body,:category_id) if params[:note]

215.     end

216.     To use these params, we need to modify the create and update methods to accept parameters through the note_params variable:

217.   app/controllers/ap1/v1/notes_controller.rb

218.   # POST /notes

219.     # POST /notes.json

220.     def create

221.       @note = Note.new(note_params)


223.       if @note.save

224.         render json: @note, status: :created, location: @note

225.       else

226.         render json: @note.errors, status: :unprocessable_entity

227.       end

228.     end


230.     # PATCH/PUT /notes/1

231.     # PATCH/PUT /notes/1.json

232.     def update



235.       if @note.update(note_params)

236.         head :no_content

237.       else

238.         render json: @note.errors, status: :unprocessable_entity

239.       end

240.     end

241.     We will add a module in order to serialize the JSON data input and output of our API. It adds a more object-oriented approach as opposed to a hash-oriented approach to JSON:

242.   Gemfile

243.   gem "active_model_serializers"


245.   notely$bundle install

246.     We already have a model before adding the serializer, hence we will run the following command to generate a serializer for our existing model:

247.   notely$ rails g serializer note

248.     create  app/serializers/note_serializer.rb

249.     We will modify our serializer in order to accept all the attributes:

250.   app/serializers/note_serializer.rb


252.   class NoteSerializer < ActiveModel::Serializer

253.     attributes :id, :title, :body

254.   end

Objective complete – mini debriefing

In this task, we generated a Rails API project with Rails 4.1 and MySQL as the database. We can use any database we want, such as PostgreSQL and MongoDB. However, we chose MySQL for the sake of simplicity of demonstration. It would be noteworthy to know that in the application generated by Rails API, the application controller inherits from ActionController::API instead of the ActionController::base class:

class ApplicationController < ActionController::API


We also developed a skeleton to create the notes. We created a very basic API with this version. We bridged the gap between Rails 4 and Rails API to accept the parameters.

We used the active serializer module in order to serialize JSON, generating a serializer for the existing model. We added attributes for note to the serializer using the following method:

def note_params

  params.require(:note).permit(:title, :body,:category_id) if params[:note]


There are several JSON template options such as JBuilder and Rabl to name a couple. JBuilder comes by default with Rails, whereas Rabl can be added as a gem. One of the primary reasons to select the ActiveModel serializer over the other two is performance. As the dataset increases in size, the process of JSON generation slows down, thus affecting the overall performance of the API. The ActiveModel serializer, however, has been known to perform better than these libraries.

In the preceding method, we have enabled the posting of params only if they are present. We also added a serializer to package this JSON hash as an object and posted it to the params method. This attribute definition will make params an attribute of an object called note, which is posted to params[:note]:

class NoteSerializer < ActiveModel::Serializer

  attributes :id, :title, :body


When we visit notes.json, we will see the following code snippet:



    {"id":1,"title":"First Note","body":"Buy a new ram"},

    {"id":2,"title":"Second Note","body":"Buy Macbook pro"}



The complete URL looks like http://localhost:3000/api/v1/notes.json. Since we have created a controller namespace and added a version, we created versioned endpoints for our API. When we write Version 2, we can simply create a new namespace for v2 with the same controllers, methods, and updated code. In that way, we can keep both versions of the API live parallely:

  namespace :api do

    namespace :v1 do

      resources :notes



  resources :notes

In order to make this namespace work in the controller, we need to define our controller like a module. By default, Rails treats the path elements as modules.

From time to time, we are expected to make major changes in our application. As and when our application is updated, there are several changes it goes through, such as the structure of data, changes in fields, and so on. Sometimes the changes are not backward compatible and there are clients already using our application. In order for them to keep using our application effectively, we need a new version of the API, rather than changing the entire API entity itself.

Arranging notes category wise

In our application, users will need to arrange their notes category wise. A category will act as a search filter to keep the notes. We will add an association via the models. This task deals with the creation of the category model and its association with the note model.

Engage thrusters

We will now add categories to our note application:

1.    We will first create a category model for our application using the following command:

2.  notely$ rails g model category title:string

3.    The migration file generated looks as follows:

4.  class CreateCategories < ActiveRecord::Migration

5.    def change

6.      create_table :categories do |t|

7.        t.string :title


9.        t.timestamps

10.    end

11.  end



14.:~/notely$bundle exec rake db:migrate

15.          We will save the category title for now and we will try to add categories using our Rails console:

16.:~/notely$ rails c

17.Loading development environment (Rails 4.0.1) :001 > category = Category.new

19. => #<Category id: nil, title: nil, created_at: nil, updated_at: nil> :002 > category.title = "Personal"

21. => "Personal" :003 > category.save

23.   (0.4ms)  BEGIN

24.  SQL (38.1ms)  INSERT INTO `categories` (`created_at`, `title`, `updated_at`) VALUES ('2013-12-26 00:02:21', 'Personal', '2013-12-26 00:02:21')

25.   (53.9ms)  COMMIT

26. => true

27.          We will then add a has_many relation in the category model as follows:



30.class Category < ActiveRecord::Base

31.  has_many :notes


32.          Likewise, we will add a belongs_to relationship in our note model:



35.class Note < ActiveRecord::Base

36.  belongs_to :category 


37.          In order for the association to work, we will add a category_id column to our note table:

38.notely$ rails g migration add_category_id_to_notes category_id:integer

39.      invoke  active_record

40.      create    db/migrate/20131226000927_add_category_id_to_notes.rb

41.          The migration will look as follows:

42.db/migrate$ nano 20131214094532_add_category_id_to_notes.rb

43.class AddCategoryIdToNotes < ActiveRecord::Migration

44.  def change

45.    add_column :notes, :category_id, :integer

46.  end

47.endFinally run bundle exec rake db:migratedb:migrate to generate the table

notely$ bundle exec rake db:migrate

48.          So, our schema now looks like the following:


50.create_table "categories", force: true do |t|

51.    t.string   "title"

52.    t.datetime "created_at"

53.    t.datetime "updated_at"

54.  end


56.  create_table "notes", force: true do |t|

57.    t.string   "title"

58.    t.string   "body"

59.    t.datetime "created_at"

60.    t.datetime "updated_at"

61.    t.integer  "category_id"


62.          Now that we have added the category_id column, we will also have to update our serializer:



65.class NoteSerializer < ActiveModel::Serializer

66.  attributes :id, :title, :body, :category_id


Objective complete – mini debriefing

In the preceding task, we created a category model. We created an association between the category and note models. The association is such that a category has many notes and a note belongs to a category. We also created the required migrations and added the respective fields to the database.

Sending join data via JSON

In the previous tasks, we created notes and then associated them with the categories. Now, we will customize our serializer class in order to work with the association. We will also use a REST client to see how to get data and post data to our API. Of course, using it in the client is the best way; however, in order to see if the data is being inserted correctly or not, we need a command-line interface. We will use the REST client's command-line interface to interact with our API.

Engage thrusters

We will serialize our data and prepare to get, post, and put the data by performing the following steps:

1.    We will first add the association data to our ActiveModel serializer. We will add the has_one :category method and embed it in order to automatically add category_id to our JSON object as follows:

2.  app/serializers/note_serializer.rb

3.  class NoteSerializer < ActiveModel::Serializer

4.    embed :id


6.    attributes :id, :title, :body

7.    has_one :category


8.    When we navigate to our notes URL, we get the following JSON values with category_id appended to them. An alternate way to check the response is via the curl command:

9.  http://localhost:3000/api/v1/notes


11.  [

12.    {"id":1,"title":"First Note","body":"Buy a new ram","category_id":1},

13.    {"id":2,"title":"Second Note","body":"Buy Macbook pro","category_id":2}

14.  ]


15.          The first step to start testing whether our API works is to install the REST client:

16.$ gem install rest-client

17.          The REST client is accessible as a command-line tool. So, we will open our interactive Ruby shell and try calling our API:

18.$ irb :001 > require 'rubygems'

20. => false :002 > require 'rest-client'

22. => true :004 > response = RestClient.get 'http://localhost:3000/api/v1/notes'

24. => "{\"notes\":[{\"id\":1,\"title\":\"First Note\",\"body\":\"Buy a new ram\",\"category_id\":1},{\"id\":2,\"title\":\"Second Note\",\"body\":\"Buy Macbook pro\",\"category_id\":2},{\"id\":3,\"title\":"note",\"body\":"study",\"category_id\":1}"

25.          We stored our response in a variable so that we can see some of the common attributes of the response. We can return the reponse code and headers for the sake of testing: :005 > response.code

27. => 200 :006 > response.headers

 => {:x_frame_options=>"SAMEORIGIN", :x_xss_protection=>"1; mode=block", :x_content_type_options=>"nosniff", :x_ua_compatible=>"chrome=1", :content_type=>"application/json; charset=utf-8", :etag=>"\"852ad43f6964fa588ce190c8fc8c7239\"", :cache_control=>"max-age=0, private, must-revalidate", :x_request_id=>"20bed29f-bffb-4743-9887-f427686c7187", :x_runtime=>"0.030178", :transfer_encoding=>"chunked"}

29.          We will try posting our first note using the API: :013 > RestClient.post('http://localhost:3000/api/v1/notes', {:note => {:title => 'test', :body => 'body', :category_id => 2}})

31. =>


33.          In our server log, we can see the post request and the 201 response code:

34.Started POST "/api/v1/notes" for at 2013-12-27 07:43:04 +0800

35.Processing by Api::V1::NotesController#create as XML

36.  Parameters: {"note"=>{"title"=>"test", "body"=>"body", "category_id"=>"2"}}

37.   (0.3ms)  BEGIN

38.  SQL (2.8ms)  INSERT INTO `notes` (`body`, `category_id`, `created_at`, `title`, `updated_at`) VALUES ('body', 2, '2013-12-26 23:43:04', 'test', '2013-12-26 23:43:04')

39.   (49.0ms)  COMMIT

40.  Category Load (0.5ms)  SELECT `categories`.* FROM `categories` WHERE `categories`.`id` = 2 ORDER BY `categories`.`id` ASC LIMIT 1

41.Completed 201 Created in 131ms (Views: 10.3ms | ActiveRecord: 52.6ms)

42.          Once we see the 201 response code, we can check if the value has been inserted successfully in our database or not:

43.$ rails c

44.Loading development environment (Rails 4.0.1) :001 > Note.last

46.  Note Load (0.8ms)  SELECT `notes`.* FROM `notes` ORDER BY `notes`.`id` DESC LIMIT 1

47. => #<Note id: 8, title: "test", body: "body", created_at: "2013-12-26 23:43:04", updated_at: "2013-12-26 23:43:04", category_id: 2>

48.          We will now check both the index and show methods using our REST client: :010 > RestClient.get 'http://localhost:3000/api/v1/notes'

50. => "{\"notes\":[{\"id\":1,\"title\":\"First Note\",\"body\":\"Buy a new ram\",\"category_id\":1},{\"id\":2,\"title\":\"Second Note\",\"body\":\"Buy Macbook pro\",\"category_id\":2},{\"id\":3,\"title\":null,\"body\":null,\"category_id\":null},{\"id\":4,\"title\":\"test\",\"body\":\"body\",\"category_id\":2}]}"

51.          In our show method, we can directly call our resource ID: :011 > RestClient.get 'http://localhost:3000/api/v1/notes/1'

53. => "{\"note\":{\"id\":1,\"title\":\"First Note\",\"body\":\"Buy a new ram\",\"category_id\":1}}"  

Objective complete – mini debriefing

In this task, we modified our serializer to add the note and category association to it. As you will notice, the association here looks slightly different from our traditional model association:

class Note < ActiveRecord::Base

  belongs_to :category 


class Category < ActiveRecord::Base

  has_many :notes


In the case of the serializer, the same association looks as follows:

class NoteSerializer < ActiveModel::Serializer

  embed :id

  attributes :id, :title, :body

  has_one :category


As opposed to models, serializers are not concerned with the ownership of a record and rather focus on multiplicity. This means if many notes have one category, the serializer still treats it as a multiple record with the value of 1. So, belongs_to makes way for has_onein serializers; it is just a different perspective to the same concept of association. The embed :id parameter will give access to the category_id field so that we do not have to worry about the attributes explicitly. The associated data is also embedded inside our JSON hash. Hence, the serializer will generate a nested JSON hash for an embedded association data. The following code snippet is the JSON object that is returned when we access localhost:3000/api/v1/notes:

Objective complete – mini debriefing

Embedding an association also gives the advantage of access to the entire category object from the note. This removes the need for another serializer for the categories. Also, in our use case, we have the has_one association. In case we want a has_many association, the embed will change as follows:

embed :ids

has_many :categories

This will supply an array of category_id fields to each record of notes. Now that we have formatted our data in the JSON format, we can secure our API.

Creating an OAuth2 provider

The most important reason underlying API development is the creation of the developer community. The applications contributed by different developers not only increase the popularity of the app, but also bring out several creative things people can do with the data; Twitter API is one such example. People have made amazing desktop clients and mobile apps that analyze tweets for trends and sentiments based on data. However, all these applications need to be genuine and should not spam the users. In order to avoid that, we will allow only OAuth-authorized applications to build clients for our API. Therefore, we will have to create an OAuth2 provider.

Prepare for lift off

Before we start working on this task, we will install devise. For the most part, the devise installation is pretty standard. In this case, we will use devise with warden as we will allow token-based authentication via warden using our doorkeeper gem:


gem 'devise'

gem 'warden'

However, as Rails API removes the middleware layer and devise has some middleware dependencies, we will have to include them in our application controller:


class ApplicationController < ActionController::API

  include ActionController::MimeResponds

  include ActionController::ImplicitRender


ActionController::MimeResponds includes the respond_to and respond_with methods of Rails. ActionController::ImplicitRender includes methods such as default_render, method_for_action, and send_action. We also need to include the middleware flash module for our application to work. Doorkeeper uses Flash to display notices and alerts as shown in the following code:


module Notely

  class Application < Rails::Application

    config.middleware.use ActionDispatch::Flash



Engage thrusters

We will make our application an OAuth2 provider in the following steps:

1.    We will use the doorkeeper gem to create our OAuth2 provider:

2.  Gemfile

3.  gem 'doorkeeper', '~> 0.7.0'


5.  notely$ bundle install

6.    We will run the doorkeeper generator once the gem is bundled successfully:

7.  $ rails generate doorkeeper:install

8.        create  config/initializers/doorkeeper.rb

9.        create  config/locales/doorkeeper.en.yml

10.       route  use_doorkeeper

11.          This will create an initializer, a locale file, and add a route for endpoints in our application.

12.          The doorkeeper gem also generates a migration. It creates a table to store OAuth access tokens and access grants:

13.notely$ rails generate doorkeeper:migration

14.      create  db/migrate/20131222100518_create_doorkeeper_tables.rb

15.          Now, create tables with the rake task:

16.notely$ rake db:migrate

17.==  CreateDoorkeeperTables: migrating =========================================

18.-- create_table(:oauth_applications)

19.   -> 0.1671s

20.-- add_index(:oauth_applications, :uid, {:unique=>true})

21.   -> 0.2556s

22.-- create_table(:oauth_access_grants)

23.   -> 0.1107s

24.-- add_index(:oauth_access_grants, :token, {:unique=>true})

25.   -> 0.2004s

26.-- create_table(:oauth_access_tokens)

27.   -> 0.1109s

28.-- add_index(:oauth_access_tokens, :token, {:unique=>true})

29.   -> 0.2001s

30.-- add_index(:oauth_access_tokens, :resource_owner_id)

31.   -> 0.2004s

32.-- add_index(:oauth_access_tokens, :refresh_token, {:unique=>true})

33.   -> 0.1899s

34.==  CreateDoorkeeperTables: migrated (1.4364s) ================================

35.          We will need to modify the initializer created here and make it use warden in order to access the user's resource from devise's current_user method:



38.Doorkeeper.configure do

39.  orm :active_record



42.  resource_owner_authenticator do


44.    current_user || warden.authenticate!(:scope => :user)

45.  end




Be sure to comment out or delete the following line from the code, else it will raise an error during execution:

#raise "Please configure doorkeeper resource_owner_authenticator block located in #{__FILE__}"

47.          We need to create a method to access the resource for doorkeeper to identify whether the logged-in user is authenticated against a valid application or not. This means it defines the owner of access_token which our application returns to each user:



50.class ApplicationController < ActionController::API

51.  include ActionController::MimeResponds

52.  include ActionController::ImplicitRender


54.  def current_resource_owner

55.    User.find(doorkeeper_token.resource_owner_id) if doorkeeper_token

56.  end


57.          We can now protect our API methods using the doorkeeper_for method:

58. app/controllers/api/v1/notes_controller.rb

59.class NotesController < ApplicationController

60.  before_action :set_page, only: [:show, :edit, :update, :destroy]

 doorkeeper_for :index, :show, :update, :create

61.          However, different methods require different levels of access. In order to abstract the different access levels, the doorkeeper gem has scopes. When a client requests for access, allowed actions are displayed. So first, we will have to enable the scopes in ourdoorkeeper initializer:



64.Doorkeeper.configure do


66.  orm :active_record



69.  resource_owner_authenticator do


71.    current_user || warden.authenticate!(:scope => :user)

72.  end



75.  default_scopes  :public

76.  optional_scopes :write, :update


77.          We have defined the following two scopes:

·        Public: This scope is for all the data that is publicly available

·        Write and update: This scope is only for users who are authenticated against the API

78.          We need to add these scopes to the controller to bring them into play:


80.class NotesController < ApplicationController

81.    before_action :set_page, only: [:show, :edit, :update, :destroy]

82.    doorkeeper_for :index, :show,    :scopes => [:public]

83.    doorkeeper_for :update, :create, :scopes => [:write, :update, :destroy]

Objective complete – mini debriefing

In the preceding task, we first prepared our application with devise and added some middleware components for devise and doorkeeper to function properly. We loaded this in our application_controller.rb and application.rb files. The doorkeeper gem is a solution to make our application an OAuth2 provider. We first installed and generated an initializer for doorkeeper.

We defined the object-relational modeling for the application. It even supports different versions of Mongoid. In our case, we use active_record. Hence, we will define it as follows:


Doorkeeper.configure do

  orm :active_record

We then added resource_owner_authenticator, which is where we connect devise and doorkeeper. We made doorkeeper use the current_user method of devise and used warden to connect to the devise methods for authentication:

  # This block will be called to check whether the resource owner is authenticated or not.

  resource_owner_authenticator do

    #raise "Please configure doorkeeper resource_owner_authenticator block located in #{__FILE__}"

    current_user || warden.authenticate!(:scope => :user)



We added current_resource_owner to check for the owner of the doorkeeper resource:

def current_resource_owner

    User.find(doorkeeper_token.resource_owner_id) if doorkeeper_token


Under the hood, doorkeeper_token accesses the token generated upon a successful authentication request and returns it:

def doorkeeper_token

  methods = Doorkeeper.configuration.access_token_methods

  @token ||= OAuth::Token.authenticate request, *methods


We finally added scopes so that we can protect the resource and give limited access to the different types of users based on their roles and ownership. For public methods, such as index and show, we defined a scope called public. We can define this for users who want to just read without logging in. For users who want to create notes, we added scopes called write and update. In order to activate these scopes, we added them as a filter method, doorkeeper_for in our controller so that they are checked before the methods are executed.

Generating API keys

Doorkeeper is a complete solution for API authorization as well as app management using the OAuth2 protocol. In the previous task, we added doorkeeper and configured it to our needs. In this task, we will see how to generate API keys and do some final integration with devise for authentication. Only logged-in users can create applications. This is a use case for when we want to give freedom to several developers to create applications using our API.

Engage thrusters

In the following steps, we will add the devise layer above doorkeeper and generate API keys for the first time:

1.    We need to generate the polymorphic association and addition to the application owner:

2.  notely$rails generate doorkeeper:application_owner

3.      create  db/migrate/20131228141233_add_owner_to_application.rb'

4.    Run the migration:

5.  notely$ rake db:migrate

6.  ==  AddOwnerToApplication: migrating ==========================================

7.  -- add_column(:oauth_applications, :owner_id, :integer, {:null=>true})

8.     -> 0.1840s

9.  -- add_column(:oauth_applications, :owner_type, :string, {:null=>true})

10.   -> 0.2002s

11.-- add_index(:oauth_applications, [:owner_id, :owner_type])

12.   -> 0.4107s

13.==  AddOwnerToApplication: migrated (0.7954s) =================================

14.          The table now looks like the following:

15.  create_table "oauth_applications", force: true do |t|

16.    t.string   "name",                      null: false

17.    t.string   "uid",                       null: false

18.    t.string   "secret",                    null: false

19.    t.string   "redirect_uri", limit: 2048, null: false

20.    t.datetime "created_at"

21.    t.datetime "updated_at"

22.    t.integer  "owner_id"

23.    t.string   "owner_type"


24.          We will add enable_application_owner in order to enable the ownership of created applications. This is false by default because we would not want the application owner to confirm his/her membership to use the application. If the value is true, the owner will be asked to authenticate against the application just like other users:

25.Doorkeeper.configure do


27.  orm :active_record



30.  resource_owner_authenticator do


32.    current_user || warden.authenticate!(:scope => :user)

33.  end


35.  enable_application_owner :confirmation => false



38.  default_scopes  :public

39.  optional_scopes :write, :update


40.          In order to access the current_user object, we need to authenticate and log in before we create the apps:

41. class User < ActiveRecord::Base

42.  # Include default devise modules. Others available are:

43.  # :confirmable, :lockable, :timeoutable and :omniauthable

44.    devise :database_authenticatable, :registerable,

45.          :recoverable, :rememberable, :trackable, :validatable

46.    has_many :oauth_applications, class_name: 'Doorkeeper::Application', as: :owner


47.          We will have to modify our controller so that we can access current_user in it. As doorkeeper is a Rails engine, we will create a folder called OAuth inside our controllers and copy the application controller to the folder:

48.notely$mkdir oauth

49.notely$cd oauth



52.class Oauth::ApplicationsController < Doorkeeper::ApplicationsController

53.  before_filter :authenticate_user!


55.  def index

56.    @applications = current_user.oauth_applications

57.  end


59.  # only needed if each application must have owner

60.  def create

61.    @application = Doorkeeper::Application.new(application_params)

62.    @application.owner = current_user if Doorkeeper.configuration.confirm_application_owner?

63.    if @application.save

64.      flash[:notice] = I18n.t(:notice, :scope => [:doorkeeper, :flash, :applications, :create])

65.      respond_with [:oauth, @application]

66.    else

67.      render :new

68.    end

69.  end



71.          We will now boot our server and log in. In order to create the OAuth2 application, we will have to browse to localhost:3000/oauth/applications. We will be presented with the application management dashboard as shown in the following screenshot:

Engage thrusters

72.          For example, in all OAuth2 providers, we will have to add the application name and callback URL for our application. It is better to enter either a real and valid domain name or a domain that resolves at localhost (lvh.me).

Engage thrusters

73.          Lastly, we will add a devise authentication so that a user needs to pass the username and password to get an access token:



76.resource_owner_from_credentials do |routes|


78.    request.params[:user] = {:email => request.params[:username], :password => request.params[:password]}


80.    request.env["devise.allow_params_authentication"] = true


82.    request.env["warden"].authenticate!(:scope => :user)



Objective complete – mini debriefing

Up until now, we have created a devise-based authentication and doorkeeper authorization for the applications. However, we had to allow the users to create authorizable applications. The doorkeeper project resides on GitHub(https://github.com/applicake/doorkeeper) and the documentation can be found at the project wiki (https://github.com/applicake/doorkeeper/wiki). There are several other tutorials that can be found on it, including ones to build a client application.

Doorkeeper allows us to create ownership for the applications that developers want to create. We ran a generator task in doorkeeper to create the migration for that:

notely$rails generate doorkeeper:application_owner

This generates the following migration by adding owner_id and owner_type to the oauth_applications table:

class AddOwnerToApplication < ActiveRecord::Migration

  def change

    add_column :oauth_applications, :owner_id, :integer, :null => true

    add_column :oauth_applications, :owner_type, :string, :null => true

    add_index :oauth_applications, [:owner_id, :owner_type]



We then enabled application ownership. We have set confirmation to false so that the application owner does not need to connect and confirm the app before using it.

enable_application_owner :confirmation => false

If we change this to true, then even the application owner will have to grant access to the application in order to use it.

We then created an association between the user model and doorkeeper's oauth_applications model:

 has_many :oauth_applications, class_name: 'Doorkeeper::Application', as: :owner

After the association, we had to make sure a logged-in user creates the application, hence we added before_filter. We also used the current_user method of devise to call all the applications by a particular user in the index page:

  before_filter :authenticate_user!

  def index

    @applications = current_user.oauth_applications


Our create method also ensures that every application has to have an owner using the confirm_application_owner? method. This is a use case for when we want a lot of users to use our application:

def create

  @application = Doorkeeper::Application.new(application_params)

  @application.owner = current_user if Doorkeeper.configuration.confirm_application_owner?

    if @application.save

      flash[:notice] = I18n.t(:notice, :scope => [:doorkeeper, :flash, :applications, :create])

      respond_with [:oauth, @application]


      render :new


Then we went ahead and created an application. An application ID and secret are created upon submitting the application form. The callback URL is generally a valid URL because the application has to return to it after authorization:

Objective complete – mini debriefing

In order to test our API, we will use curl and send a request. This request includes client_id, client_secret, username, and password and is formatted as follows:

$curl -i http://localhost:3000/oauth/token \

     -F grant_type=password \

     -F client_id="5885177a47013eda93464fb764567d1ebe8ba411001e798fc1745d22e839ce36" \

     -F client_secret="8f131eceab62a2bdf149f74a33c0bd24ad616d84617ab10673c1a433ef81c7ba" \

     -F username="saurabh.a.bhatia@gmail.com" \

     -F password="safew123"

We can view the response in the following screenshot:

Objective complete – mini debriefing

The response of the API includes access_token, token_type, and expires_in (expiry time). This means our application is successfully authenticating as well as authorizing using OAuth:







This check is done using the rule that we added in the previous step:

resource_owner_from_credentials do |routes|

The previous (resource_owner_from_credentials) method matches the supplied credentials using the devise user model.

Securing the application

Security is one of the primary concerns of an API application. We have already provided some level of security with authentication and authorization. However, we still need to add extra layers of security to our application. Doorkeeper and warden allow token-based authentication, and hence a user has to have an authentication token. Also, the application is authenticated against the application ID and secret.

Engage thrusters

We will now add some security-related tricks to our application by performing the following steps:

1.    The first level of security we will provide is against session fixation. In our devise initializer, we need to add the following lines:

2.  config/initializers/devise.rb


4.  Warden::Manager.after_authentication do |record, warden, options|

5.    warden.request.session.try(:delete, :_csrf_token)


6.    We will now set up the session timeout in our application so that the session is deleted after the specified time interval:

7.  app/models/user.rb


9.  class User < ActiveRecord::Base

10.  devise :database_authenticatable, :registerable,

11.          :recoverable, :rememberable, :trackable,:validatable, :timeoutable, :timeout_in => 15.minutes


13.  has_many :oauth_applications, class_name: 'Doorkeeper::Application', as: :owner


14.          SQL injection attacks are pretty common in web applications. However, Rails provides enough protection against SQL injection. Rails already provides one level of protection in the controllers by whitelisting parameters:



17.def note_params

18.  params.require(:note).permit(:title, :body,:category_id) if    params[:note]


19.          In case we need to pass a parameter, we need to pass it as a string:

Note.where(:title => "'#{params[:title]}'")

20.          The right way to avoid SQL injection using the parameter is as follows:

Note.where("title=?", title)

Objective complete – mini debriefing

Security is an extremely critical aspect of our applications in today's world. In this task, we looked at some of the ways Rails already provides security to the application by default, and some other ways in which we can secure our application.

The first thing we looked at was session fixation. Wikipedia defines session fixation as follows:

In computer network security, session fixation attacks are an attempt to exploit the vulnerability of a system, which allows one person to fixate another person's session identifier. Most session fixation attacks are web based, and most rely on session identifiers being accepted from URLs or POST data.

Devise out of the box is quite secure. However, this scenario can occur in the following two cases:

·        When the attacker uses subdomain cookies to enter the target session

·        When the attacker exploits the same Wi-Fi network for fixation

In order to avoid this, we delete the following unique session CSRF token as soon as the authentication is complete:

  warden.request.session.try(:delete, :_csrf_token)

The preceding line of code will clear the CSRF token. So, if an attacker is trying to steal the token, they are not able to, and hence the session is secure.

Another way that we looked at was timing out our sessions. Sessions are most susceptible to attack when they have some idle time on them. In order to avoid these attacks, we can clear the session. The timeoutable module in devise allows us to define when to expire the session:

:timeoutable, :timeout_in => 15.minutes

We defined in our application that the session should expire if it is idle for 15 minutes. This setting should use a much higher value in a real-world application because we would not want a user to log in again if they are idle for 15 minutes.

Lastly, we secured our application against SQL injection. Rails has been vulnerable to SQL injections owing to the mass assignment parameters in 3.2.x Versions, and there were serious security concerns related to it. Rails 4 sanitizes the parameters out of the box by using the standard blacklisting and whitelisting technique. Only the whitelisted parameters are allowed to pass to the controller. However, the nature of injection attacks is such that people can still insert malicious SQL statements inside params and allow them to be passed inside the query string.

Wikipedia defines SQL injection as follows:

SQL injection is a code injection technique used to attack data-driven applications, in which malicious SQL statements are inserted into an entry field for execution.

In Rails, a lot of times parameters are directly passed from a query string to the active record query interface using the params[:title] format. This makes the SQL statement vulnerable as someone can pass a string with an SQL statement, such as OR or AND, and execute SQL inside it.

We rephrased this query to a different format. We will first pass the SQL statement into a variable. Then, instead of directly passing the query string, we will sanitize the variable and pass it to a query as follows:

Note.where("title=?", title)

Another way is to pass the variable through the sanitize_sql() method before passing it to the query.

Also, in a real-world application, it is highly advisable to use an SSL certificate in order to provide secure access to your server, especially for transactions that involve passing your application's application_id and application_secret.

Mission accomplished

We have successfully created an API only application with an OAuth2 provider. We also looked at various security aspects in this project.

Some of the areas we covered in this project are as follows:

·        We used the Rails API gem to create an API only application that does not contain rack middleware modules and frontend modules.

·        We adapted some of the methods to Rails 4, for example, strong parameters.

·        We used a JSON serializer to create clean JSON APIs that are interactive. We can read, write, and update the data using JSON.

·        We also looked at how associations work in the serializer and how it is different from the regular models.

·        We also saw how to read and post to the API using a REST client.

·        Once we had a fully functional API in place, we used the doorkeeper gem in order to protect these API methods.

·        We saw how doorkeeper in conjunction with devise turned the application into a full-fledged OAuth2 provider, thus providing it with an authentication as well as a conditional authorization framework.

·        We created our own application using the newly created OAuth framework.

·        We also worked on how to secure our application from different types of attacks.

Hotshot challenges

We have our API in place and a lot of functionalities to play around with. However, we would still like to take things to the next level with the following exercises:

·        Allow and delete notes in the API and delete a note using the REST client.

·        A different mime type to the API post method. By this I mean that you should allow a user to post an image using the post method in our API.

·        Make sure that this method works only with Ajax requests using doorkeeper.

·        Create a client application using HTTParty to read the notes of a user.

·        Log in and authorize using the devise token authentication and the OAuth2 provider.