Protecting data on AWS cloud using powerful encryption techniques

Encryption is a process used to convert plain text data into Ciphertext and thus protecting information from getting read by unauthorized users. Data can be encrypted both while travelling over the Internet or when it is stored on servers. Encryption is used in the cloud to safeguard sensitive information stored and travelling through internal networks, the Internet, and wireless mobile devices. Encryption in the cloud is particularly important for data security because the data is outside of the enterprise and general information security controls are not applicable in the cloud. It is also commonly used for compliance with industry regulations such as HIPAA and PCI DSS.

Encryption is a powerful mechanism for safeguarding an organization’s data and information both in-premise and on the cloud especially with the increasing use of Cloud based Software As a Service (SaaS) applications.

In this blog post, we will look at encryption strategies available when using Amazon Web Services (AWS) for cloud based storage and computing and achieve powerful safeguarding of our data.  

Using data encryption in AWS

There are two options to encrypt data stored on AWS S3; Client-Side encryption and Server-Side encryption. Client-side encryption can be used for encrypting data before it is sent over the network to AWS. Server-side encryption is for encrypting data at rest while on AWS. Below we take a look at both the options below and how both techniques can be combined to achieve stronger data protection on AWS.  

Client-side encryption

AWS client-side encryption is used for encrypting data before it is sent over the network to AWS. The entire process of encryption is managed by the client application (written in Ruby in this case) and the plaintext data and the master encryption key never leaves the client application. Here is how the process works:

  • AWS generates a one-time symmetric key (data encryption key) per object to encrypt data.
  • That symmetric key is then encrypted using user provided master encryption key and stored alongside data in S3.
  • The master key and unencrypted data never leave the application environment.

 

 

For this demo we are using AWS SDK for Ruby. We use client-side encryption with AES-256-CBC cipher (more about AES here). While decrypting the data, base64 encoded master key provided during encryption has to be provided by the application or an AWS service for decryption of data.  

 

Server-side encryption

Server-side encryption is used for encrypting data at rest. AWS provides three server-side encryption options. We will be discussing the server-side encryption with customer provided encryption keys. When an object is uploaded to S3, Amazon uses the encryption key provided by customer to apply AES-256 encryption and removes the encryption key from memory. Amazon does not store the encryption key provided but instead stores an HMAC value of the key to validate future requests.

Below diagram shows client-side and server-side encryption used in combination.

 

AWS SDK for Ruby

 

 

Steps for client-side and server-side encryption using AWS SDK using Ruby

 

  • Generate random encryption keys and initialization vector (iv) for both client and server encryption
  • Save the encryption key and base64 encoded key to a file.
  • Get the encryption key (not base 64 key) from key file (for client side encryption)
  • Apply client encryption key to files and set server-side encryption to aes-256.
  • Upload files to S3.

Here is the code snippet in Ruby for the above process:

# clientServerEncryption.rb
    # 
    # Purpose: 
    #   Performs the task of creating the encryption key and then uploading the files to the appropriate S3 bucket
    #
    # Reference: 
    #   Using Client-Side Encryption for S3 in the AWS SDK for Ruby (http://bit.ly/1qBabsm)
    #
    # Author: Arpan Joshi (arpan.joshi@concur.com)
    # Reviewer: Denny Lee (denny.lee@concur.com)
    #
    # Dependencies:
    #   - AWS SDK for Ruby [gem install aws-sdk]
    #
    # -----------------------------
    # required libraries 
    # -----------------------------
    require 'aws-sdk'
    require 'openssl'
    require 'Base64'
 
    # -----------------------------
    # Manually configure
    # -----------------------------
 
    # S3 bucket connections
    s3 = AWS::S3.new(
    :region => 'S3_REGION', 
    :access_key_id => 'ACCESS_KEY_ID', 
    :secret_access_key => 'SECRET_ACCESS_KEY'
    )
    bucket = s3.buckets['S3_BUCKET_NAME']
 
    # Local Directory
    local_dir = 'PATH_TO_LOCAL_DIRECTORY'
 
    # -----------------------------
    # Generate new AES256 Cipher
    #   - generate random key Base64 encode and 
    #   - save both the key and base64 encoded keys to files
    # -----------------------------
 
    cipher = OpenSSL::Cipher::AES256.new(:CBC)
    cipher.encrypt
    key = cipher.random_key
    iv = cipher.random_iv
 
    my_key = File.open("my_key", "wb") do |keyfile|
      keyfile << key
    end
 
    base_64_key = File.open("base_64", "wb") do |base64file|
      base64file << Base64.encode64(key)
    end
    # -----------------------------
    # Upload data to S3 bucket
    #   - grab my_key encryption key from key file
    #   - apply encryption key when uploading the file to s3
    # -----------------------------
    encr_key = File.read ('my_key') # get the encryption key from key file              
    Dir.glob("#{local_dir}/*.*") do |item|
      if File.file?(item)
        #puts item
        obj_key = "path_to/target_S3_prefix/"+File.basename(item)
        obj = bucket.objects[obj_key]
        obj.write(:file => "#{local_dir}/"+File.basename(item), :encryption_key => encr_key, :server_side_encryption => 'AES256')
        puts "Encrypted and Uploaded file: =>>>  #{item}"
      end
    end

The master key (base64 encoded) is then used to decrypt the data encryption key and the encrypted data. For example, in order to load data from S3 to Redshift we use the Redshift copy command with encrypted option:

COPY table_name from 's3://bucket/object' credentials 'aws_access_key_id=;aws_secret_access_key=;master_symmetric_key='
delimiter '\t' gzip encrypted;

Loading next article