Simon Buckle's Weblog Random thoughts for random people Thu, 24 Jan 2019 09:49:28 +0000 en-US hourly 1 119589747 Accessing CloudFormation Macros from other AWS Accounts Tue, 15 Jan 2019 13:10:49 +0000 In this post you’ll learn (hopefully!) how to allow other AWS accounts to access CloudFormation macros in your account. AWS announced CloudFormation macros last year. Macros allow you to use a Lambda function to process CloudFormation templates.

I won’t be going over how macros work in detail in this post, but you can learn more about them here. I also assume you have installed the AWS CLI tools.

Let’s begin.


There are two steps to allow another account to access your macros:

  1. Grant permission to the other account to invoke the Lambda function that processes the CloudFormation template.
  2. Define the macro in the account that wants to use it.

In this scenario, we have our two protagonists Alice and Bob. (They usually work in cryptography but are currently on secondment in operations.)

Below are their respective account ids:

Account NameAccount Id

Grant Permission

Bob has written a macro that automatically sets up IPv6 for a VPC that Alice wants to use from her account.

Before Alice can start using Bob’s macro in her CloudFormation templates, Bob needs to give Alice permission to invoke the underlying Lambda function. Bob opens up a terminal and runs the following command, specifying Alice’s account id as the principal:

$ aws lambda add-permission --function-name EnableIPv6 --statement-id 987654321 --principal 987654321 --action lambda:InvokeFunction

This command adds the ‘lambda:InvokeFunction’ permission to the function policy of the Lambda function, which allows Alice to call it. (Note: you can’t modify the resource policy from the management console. You can only do so using either the CLI tools or an SDK.)

You can also grant permission to another account directly in your CloudFormation template. For example, to allow Alice to invoke his Lambda function Bob would add the following to his template:

AlicePermission:     Type: AWS::Lambda::Permission     Properties:       Action: 'lambda:InvokeFunction'       FunctionName: !GetAtt TransformFunction.Arn       Principal: '987654321'

Create the Macro

Before she can use Bob’s macro, Alice must define a macro in her account, referencing the ARN of the Lambda function in Bob’s account like so:

Resources:   Transform:     Type: AWS::CloudFormation::Macro     Properties:       Name: EnableIPv6       Description: 'Enable IPv6 in a VPC'       FunctionName: 'arn:aws:lambda:eu-west-2:123456789:function:EnableIPv6'

Alice then navigates to the CloudFormation section in the management console and creates a stack using the template above. Once the stack has been created, Alice will then be able to use the macro in her own templates.

And that’s it! The end.

Need some help with AWS? I’m a certified AWS engineer and Linux Foundation system administrator. Get in touch for a quote.

]]> 0 848
YUI Ant Compressor Task – Update Sun, 05 Apr 2015 12:45:35 +0000 As Google Code is shutting down, I’ve migrated the YUI Ant Compressor task that I wrote a while ago to GitHub. You can find it here.


]]> 0 809
Quote Wed, 05 Jun 2013 10:32:54 +0000 I came across this quote the other day and thought I would share it:

Nothing in the world can take the place of persistence. Talent will not; nothing is more common than unsuccessful men with talent. Genius will not; unrewarded genius is almost a proverb. Education will not; the world is full of educated derelicts. Persistence and determination alone are omnipotent. The slogan “press on” has solved and always will solve the problems of the human race.

– Calvin Coolidge

]]> 0 801
Introduction to VoltDB Tue, 11 Dec 2012 16:31:03 +0000 Following up on the recent tradition, or so it seems, of starting every one of my blog posts with the words, “Introduction to”, my VoltDB tutorial has (finally!) been published on the developerWorks site: Introduction to VoltDB.

The latest version of the source code that accompanies the article can be cloned from the VoltDB example project on my GitHub account here.

If you have any feedback, please leave a comment.

]]> 0 768
Introduction to Riak: Part Deux Tue, 15 May 2012 15:13:30 +0000 Part 2 of my introduction to Riak has just been published. You can view it here:

I’ve had a quick look and there appears to be an encoding issue in Listing 2. Ignore the question marks. It should read:

 $ curl -i http://localhost:8098/riak/odds/
 ...  { "odds":"", "description":"" }  

Hopefully they will have corrected it by the time you read this.

Other than that it’s more or less how I submitted it (I think). I’ll go over it in more detail later on. Let me know what you think.

]]> 0 753
Simulating Auto Increment in VoltDB Sun, 29 Apr 2012 21:15:27 +0000 Auto incrementing fields are quite useful, particularly for allocating values to primary keys. MySQL has AUTO_INCREMENT and PostgreSQL has a SERIAL data type. VoltDB has neither, nor anything remotely close to them. This brief article will show you how to simulate auto-incrementing fields in VoltDB. It assumes some knowledge of VoltDB.

VoltDB implements a subset of ANSI-standard SQL. It supports the basic CRUD operations (INSERT, SELECT, UPDATE, DELETE) but it does not have support for automatically generating unique identifiers. It is possible, however, to simulate these in VoltDB, as per this entry in the FAQ. What we can do is create a table that stores the name of the table and the current value that can be used as the unique value for, say, a given column. The schema for the table is shown below:

The next step is to create a stored procedure that, when called, will return the current value for a given table. The stored procedure will read the current value, increment it, and then return the value to the client.

The stored procedure looks like this:

import org.voltdb.*;


@ProcInfo (    partitionInfo = "IDENTIFIER.TABLE_NAME: 0",    singlePartition = true  )  public class GenerateUniqueIdentifier extends VoltProcedure {      public final SQLStmt select = new SQLStmt(      "SELECT CURRENT_VALUE FROM IDENTIFIER WHERE TABLE_NAME = ?"    );      public final SQLStmt update = new SQLStmt(      "UPDATE IDENTIFIER SET CURRENT_VALUE = CURRENT_VALUE + 1 " +      "WHERE TABLE_NAME = ?"    );      public VoltTable[] run(String tableName)      throws VoltAbortException {          voltQueueSQL(select, tableName);        VoltTable[] idResult = voltExecuteSQL();          voltQueueSQL(update, tableName);        voltExecuteSQL(true);          return idResult; // Return the current value for the table    }  }

As this procedure demonstrates, it is possible to execute multiple SQL statements from inside a stored procedure. It also highlights the advantage of having single-threaded partitions. The table is partitioned on the table name therefore the current value for a given table (each row) is stored in a single partition. As each partition is single-threaded, and stored procedures are run sequentially, there is no risk of the current value for a given table being altered between the time the procedure reads the current value and updates it; this would not be the case in a multi-threaded environment where some kind of locking would have to be used.

With the stored procedure in place, it can be called at will. The returned values can then be used in subsequent calls to other stored procedures. And that is how you can simulate auto-increment in VoltDB. At some point I will upload a complete working solution. Checkout the autoincrement branch of my example VoltDB project to play around with some working code that implements the above. The End.

]]> 7 735
Introduction to Riak Tue, 13 Mar 2012 21:25:34 +0000 Several months ago – in a galaxy far, far away – I received an email inquiring as to whether I was interested in writing a couple of articles about Riak for IBM’s developerWorks site. I was, so I did – I first wrote something about Riak a while back on this site over here. Anyway, after a bit of a wait, the first one was released into the wild today. You can read it here:

It’s mostly intact although a few paragraphs appear to have fallen by the wayside. Not really surprising as the article was supposed to be under 3000 words whereas the (final) version I submitted was quite a bit over that.

There’s a second installment but I have no idea when it will be published. Or if for that matter. I guess that may depend on the reaction to the first one 🙂

]]> 0 726
Multi-Tenancy Tue, 27 Dec 2011 11:38:18 +0000 I attended the Alfresco conference in London in the middle of November and there was a fair amount of talk about Alfresco’s cloud offering that – if it’s not already available – was due to be launched fairly soon. It will be a hosted service and will allow a single instance of Alfresco to host multiple sites (or tenants). This is usually referred to as multi-tenancy. There are a number of different approaches but the simplest one involves sharing the same database; at the database level you can think of each entry in a table, e.g. forum posts, having something like a site ID column that indicates which site the entry belongs to.

I started thinking about it and I don’t get it. I understand technically how multi-tenancy works; I just don’t see the benefits of making an application multi-tenant aware! With the advancements in virtualisation technology made in recent years, why not just get a couple of servers[1], install some virtualisation software and create a virtual instance for each site?

I can imagine that the development effort required in making an application multi-tenant aware is non-trivial – this implies a significant monetary cost in terms of the number of developers working on it and the time it takes to develop – and then there is the potential security risk of tenant data being exposed to other tenants due to bugs in the software. It could happen, you never know! Presumably you will also need to maintain a separate version of the application that doesn’t support multi-tenancy (?)

I guess upgrades would be easier – you would only have to do it once as opposed to having to upgrade each virtual instance.

It just seems to me to be much simpler to go down the virtualisation route. You don’t need to do anything to the application, all you need to do is install some virtualisation software and off you go. Obviously I’m over simplifying it but you get the idea. Perhaps I have missed something. I would be interested to know what you think.

[1] I realise it’s not fashionable nowadays to actually buy/rent your own dedicated servers as everything has to be in the “cloud”!

]]> 1 712
Yubico Java Client Changes Wed, 05 Oct 2011 06:29:29 +0000 Just a quick note. As part of the integration work I did getting YubiKey to work with Alfresco, I also added support for signatures and making validation queries in parallel to the Yubico Java client (I forked the original client) so it should now work with version 2 of the validation protocol; see this FAQ. Hopefully I didn’t fork it up!

You can grab it from my GitHub account:

If you do decide to use it, you might want to pick and choose which bits you want to pull as I have made other changes not related to the enhancements in version 2.0 of the validation protocol. Now I wonder if I’ll get my five free YubiKeys 😉

Update: I did. Just ordered them. Thanks Yubico 🙂


]]> 0 697
Two-Factor Authentication with Alfresco Thu, 29 Sep 2011 19:27:39 +0000 So what is two-factor authentication? I’ll defer that explanation to the Wikipedia page on the subject. Most systems require users to identify themselves using a username and password. The problem is that if people choose a weak password, which evidence suggests they do, suddenly your secure authentication system is not so secure. Using “something you have” in the authentication process makes it much more secure. Take cash machines as an example. If I discover your PIN number, I can only take money out of an ATM if I am in possession of your bank card. Without the card, knowing the PIN number is not going to help me steal your money.

I’ve created an Alfresco extension that implements two-factor authentication using a YubiKey.

What is a YubiKey? It’s a device that you plug into your USB port and it generates one time passwords (OTP). It’s similar to RSA’s SecurID, only a lot cheaper. So now, in addition to specifying your username and password, you also have to submit a OTP when logging in – the OTPs are validated by Yubico’s servers.

Using a key like this makes logging in a lot more secure as it is now no longer possible to log in just using a username and password. In addition, each key is tied to a particular user account – the extension takes care of this – so it’s not possible to just use any key; the user has to use the key that has been (uniquely) assigned to them. The screencast below shows how it works.

I will release the extension shortly. You can download the extension from here. I’ll be attending Alfresco DevCon in London so come and say hi and I can give you a live demo of the system. In the meantime, if you have any questions, feel free to leave a comment or send me an email.

]]> 0 680