Instead of having a dedicated login page, some sites return a 403 Forbidden HTTP status code and include the login form in an HTML body of a custom 403 page. For example, Drupal admin pages work this way. While this may seem a little odd, it works; all modern browser will display the HTML and few, if any, will note the Forbidden status.
Mechanize on the other hand raise an exception when it receives a 403 status. Fortunately, it returns the page it received as part of that exception. Here’s how to handle it:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
This code also works in the case where you don’t get a forbidden status, so it can be used generically.
For bonus points you can use the same code in a Cucumber step by changing:
(assuming you’ve set up “the admin page” in paths.rb).
1 2 3 4 5 6 7 8 9
This is fine if you want to generate a key pair once, but what if you want to do it on the fly? The Ruby OpenSSL library has support for generating key pairs:
2048 is the key size, and a good value to use for it.
What’s not obvious is how to encrypt the private key. You don’t have to encrypt it, but, if you don’t, anyone who gets a hold of the key can decrypt your data. Using an unencrypted private key gives you one layer of security (something you have - the key), encrypting it gives you an additional layer (something you know - the password).
To encrypt the private key you need a Cipher object:
Then, using the Cipher object, you convert the the key_pair to PEM format:
1 2 3
The resulting PEM strings be saved and then later fed to OpenSSL::PKey::RSA.new() or used with Strongbox.
UPDATE: Emacs.app builds fine (and has for a while) as a 64 bit application.
1 2 3 4 5
Then move nextstep/Emacs.app in to /Applications.
The developers have been actively working on Snow Leopard issues; hopefully 64 bit support is not far behind.
Scott Chacon, one of the guys behind GitHub, has released Pro Git, which, as the name suggests, is a new Git book. He’s made it available under the Creative Commons Attribution-Non Commercial-Share Alike 3.0 license, which is to say it’s free. It cover the basics you’d except, but really shines when it comes to advanced usages, customization, policy, hosting, internals, etc. It’s the resource to reach for when it’s time to loose your amateur status. And did I mention it’s free?
You can show your support by buying a print copy (the author’s link, not mine).
The key to making your code more readable with “unless” to is is to think semantics, not syntax. In spoken languages if there is more that one way to say something, there will be subtle differences in meaning. In English “unless” does not mean the same as “if not”, instead it indicates an exception or unlikely condition. For example, compare:
Unless it rains, we will go to the park.
If it’s not raining, we will go to the park.
The first sentence suggests that rain is possible, but not likely, the second that it is likely to be raining. To get the most out “unless” in your code, use it the same way.
John gives this as example of code that’s very readable with “unless”:
Simple, elegant, and anyone reading it will understand that we expect the format to be allowed. In the unlikely case that the format falls outside of what we expect we raise an exception. Where as with “if”:
is just a little bit ugly and says less about what is expected.
In his post, John argues against using “if” with “else” but I disagree. Again, I think in makes for more readable code when you are testing for an unexpected condition. Consider code that checks for an inactive user:
1 2 3 4 5
Reading the code, our expected course of action, doing stuff for the user, comes first, followed by the unlikely case that an inactive user is trying to do stuff. You could of course write:
and there’s nothing wrong with that. However, I feel that using “unless” makes it clear what is expected.
In short, if you are using “unless” as a simple macro for “if !()”, then you shouldn’t be using it at all. However, “unless” is another tool which, when used with care, can make your code clearer.
Over a year ago I wrote the wildly popular Encrypting Lots of Sensitive Data with Ruby (on Rails). At the end I said:
Clearly, this screams for a plugin; watch this space.
Well, it took a while and it turned out to be a gem, but Strongbox has arrived.
First a recap:
You have a web application and you need to encrypt the data your receive from your users. The most common form of encryption is symmetric-key encryption, where one password is used for both encryption and decryption. This works very well, but it means that everyone who enters data needs to know the password and everyone who knows the password can decrypt the data.
Enter Public-key cryptography which used one password (key) to encrypt and a different key to decrypt. This solves the problem; make the encryption password, the public key, available to your application, and keep the decryption password, the private key, well, private. Users don’t need to know or care how, they fill out a form and the data gets encrypted. One small problem, size. The most you can practically encrypt using this method is 245 bytes. Good enough for the launch codes, but not so for driving directions to the buried treasure.
No problem, if we have larger data, we simply combine the two. We generate a random password and use it to symmetrically encrypt to data. We then use the public key to encrypt the random password. To get the data back, the private key is used to decrypt the random password which is in turn used to decrypt the original data.
Got it? Good. Strongbox takes the above three paragraphs and reduces them to this:
1 2 3 4 5 6 7 8 9
OK, it’s sightly more complex. The column “secret” needs to exist in the database and be type “binary” (more on this in a bit). In additional, because we are using symmetric encryption (the default), we need two binary columns “secret_key” and “secret_iv” to store the generated symmetric key and Initialization vector (IV) (which you can think of as a second key (but it’s not) once they are encrypted with the public key.
If you are certain that the data you are encrypting won’t be larger than 245 bytes, you can use the following:
1 2 3 4 5 6
This skips the symmetric encryption, is faster, and you only need the binary “secret” column.
You’ll also need to generate a key pair. Be sure to choose a strong pass phrase, as this is the one that will decrypt everything (as always, I suggest using Diceware).
1 2 3 4 5 6 7
If you aren’t going to be decrypting data on a regular basis you might want to deploy just the public key. Extract it:
1 2 3 4
And change your model:
1 2 3 4 5
You could then have rake/Capistrano task to deploy and remove the private key as needed. Or you could limit it’s use to a separate, non-public, server.
As noted above you want your database column(s) to be binary. If your database does not have a binary type you can add the :base64 option:
1 2 3 4 5
This will convert the binary data to text using Base64. You must, must make your column type “text”. Base64 increases the length of the data it encodes by approximately 137%. Type “string” is typically 256 bytes, 245 * 1.37 = 335.65 bytes. If you use a “string” column and encrypt anything greater than 186 bytes your data will be lost.
Finally, there are two addition options for tweaking the encryption settings that you are unlikely to need:
“:symmetric_cipher” lets you change the algorithm that’s used for symmetric encryption. The default is 256 bit Advanced Encryption Standard (AES) using Cipher Block Chaining (CBC) (‘aes-256-cbc’ in OpenSSL terms). AES has been approved by the NSA for top secret information, so it’s probably good enough, but Blowfish (‘bf-cbc’) is know to work as well. Other ciphers in CBC mode should also work, but have not been tested by me. (Note that all ciphers may not be supported by your version of OpenSSL, “openssl list-cipher-commands” will provide a list.)
“:padding” allows you to change the method used to pad data encrypted with the public key. Unless you are working with legacy data, you shouldn’t need to change this. The default is “RSA_PKCS1_PADDING”, see the code if you need other options.
I am not a security expert. This software using an off the shelf encryption tool, namely OpenSSL, that has been well tested, but that is not a guarantee that this implementation doesn’t have weakness. Be sure you understand what Strongbox does, and review it for your application. A few things to keep in mind:
- Strongbox encrypts the data as it is saved, but no sooner. Be sure to use HTTPS for submitting the forms (and decrypting data!).
- If an attacker gains entry to your system the encryption should protect your data. However, they might be able to hack your code to intercept new data or, much worse, your private key password. Protect your server.
- When decrypting make sure your data isn’t cached.
And test, test, test. If there is a problem with how your data is encrypted, there is no getting it back.
One concern I have is garbage collection. If you decrypt something into a variable which then goes out of scope, how long does it hang around in memory? Can you force it out? I haven’t found much information on this; if there are any Ruby GC experts out there, share your knowledge!
I am always open to suggestions and improvement, but, to quote the License:
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Finally, I’d like to give a shout-out to thoughtbot. While this software has existed for many years, the final form of the gem was greatly inspired by Paperclip. It’s a nice example of an approach to adding complex features to an ActiveRecord attribute and how to test them. In additional, the gem sat unpublished for nearly a year because it needed test coverage and my testing was week. Then I took thoughtbot’s Advanced Ruby on Rails class which really helped me get my head around testing and TDD, and got this moving again. If you know Rails, but need to improve your processes, I highly recommend this class.
I’ve been using Emacs for more than 20 years now. I still use it just about everyday on servers I admin. However, a year or so ago I started using TextMate as my day to day editor for development on the Mac. TextMate has some nice features: a very usable project mode, code snippets, variable completion (Emacs has these as well, but that’s another post), and it’s what all of the cool kids were using.
So, I’m going to give Emacs on the Mac another shot. Not (just) to be cool, but because I can’t so much as type this post without trying to use Emacs key strokes that may not be there (curse you Wordpress for taking my Control-A!).
In the future I plan to look for Emacs equivalents for TextMate features I’ll miss. But the first question I need to answer is “Which Emacs to use?” There are a number of options, all with pluses and minuses. Since I couldn’t find much in the way of comparisons, I thought I’d better review them myself. I’ve limited this to native OS X ports of GNU Emacs. There also exists XEmacs, a major fork of GNU Emacs with Mac support but I’m not an XEmacs user. And it’s possible to run X Windows versions of Emacs under OS X, but if that’s your thing, you don’t need my help setting it up.
- Are GUI based. (You may have only seen console versions of Emacs but it’s had a GUI for at least 15 years.)
- Support common Command/Apple key bindings, so you can save files with Command-S instead of C-x C-s, open files with Command-O, cut and paste with Command-C and Command-V, and so on.
- Are integrated with the system cut buffer; if you copy, or kill (cut) text in using Emacs commands, it will be available to paste in other applications. Likewise, text copied or cut in other applications can be pasted into Emacs using it’s yank (paste) commands.
- Follow the X-Windows style of copy on select, paste on middle-button
Aquamacs is the most Mac like Emacs of the bunch. The Aquamacs developers strive to give Emacs a Mac feel. It features customizations such as a Mac-style tool bar, user friendly menus, and, in the 1.6 preview version, tabbed buffers. When using the Command/Apple keys to open or save files it uses the standard file dialogs. The distribution also includes a large number of add-on editing modes and packages, many of which have been pre-configured.
One oddity is it’s printing support, it sends the buffer to Preview for printing.
If you are new to the Emacs, then Aquamacs is probably the place to start. However, if you’ve used Emacs previously you may find that some Emacs-ness has been lost, and if you plan to use Emacs on other platforms you may find some of the differences confusing.
Interestingly, while not the official version, out of the box Carbon Emacs is Emacs to a fault; by default standard Mac key-binding are not enabled and the Command/Apple key is the meta-key. This is, in fact, the version of Emacs that ships with OS X 10.5 Leopard and is installed in “/usr/bin/emacs”. Apple does not include the .app wrapper so it only runs in the command line. In theory you could download and install just the wrapper, but it’s much easier to grab the full version from Apple
To make it more Mac like add:
to your .emacs.el which will enable the standard Command/Apple key bindings and make the Alt/Option key the meta key. Thus configured it strikes a good balance between being Emacs and being a Mac application.
Like Aquamacs, Carbon Emacs ships with additional pre-configured editing modes and packages. Unlike Aquamacs it doesn’t add much in the way of Mac-ish features. It also has the best printing support, being the only one of the three to use the native print dialog. Carbon Emacs is actively developed, but, because of the time needed to integrate it’s changes, it may lag a bit behind the official version.
Carbon Emacs is a good choice for people who want a “standard” Emacs with a Mac feel to it and a good collection of pre-configured add-ons.
Emacs.App (or EmacsApp) is the official version of Gnu Emacs for the Mac. It’s based on the NeXTstep / OpenStep / Cocoa API and is very similar in look and feel to Carbon Emacs. Naturally it only includes the standard packages and modes that come with official versions of Emacs.
The major downside is that Emacs.App is not in the current stable Emacs release (22.3). It will be included in Emacs 23, due (or predicted for) later in 2009, but until then it only available in the developer “edge” version through CVS. This creates a further issue in that, while there are binary distributions available they tend to be out of date, so you’ll have to build it from source. Fortunately this is very straight forward:
1 2 3 4 5 6
(The “make install” step builds the .app wrapper, it doesn’t actually install anything.)
Emacs.app is also available from MacPorts and Fink.
When all is said and done Emacs.App is Emacs; if you want to be be up to date with the current version, then this is the one to use. However, you will need to install, configure, and maintain any additional modes and packages you want to use.
In short, choose Aquamacs if you want a more Mac-Like Emacs. For the Emacs experience with less effort, use Carbon Emacs. For the most current Emacs, with all of the risks and rewards of using edge software, plus a little more work, use Emacs.App.
Myself, I’m going to go with Emacs.App. It’s closest to the versions I use on my servers, willing to beta-test, and I’m comfortable installing and configuring additional packages.
So, you have a column in your database you can’t update after the record is created. Not don’t want to update, but can’t. Specifically, you might have a column that is protected by a trigger, which will cause an error if that column is included in a update. How do you prevent ActiveRecord from trying to update that column?
Prior to Rails 2.0, ActiveRecord will always generate an SQL UPDATE statement that includes all of the attributes in the model, even if they hadn’t changed.
1 2 3 4 5 6 7
If “sku” happens to be read-only, the update will fail, and so will your app.
The right way to fix this is to upgrade to Rails 2.x. Starting 2.0 you can use attr_readonly which (silently) removes the attribute from the UPDATE statement.
1 2 3 4 5 6 7
And, starting with 2.1, ActiveRecord only updates attributes that have been changed. As long as you don’t change the value of an attribute, it won’t be included in the UPDATE statement.
1 2 3 4 5 6
(Obviously, it’s better to explicitly mark an attribute as read-only then to depend on this behavior.)
But, what if you are working with a pre 2.X version of Rail? As I said above, ActiveRecord generates the UPDATE statement based on the attributes in the model. The trick, or should I say ugly hack, is to load the record with only the fields you want to update using :select. This way, when the UPDATE is generate it will only include those attributes that were loaded into the record.
1 2 3 4 5 6
When doing this, you need to include the “id” column (or whatever your primary key is) in the select. Also note that while this will work with find_by methods, find_or_initialize_by methods do not take the :select option.
Yup, it’s ugly but, it does, in fact, work.
Rick Olson’s attachment_fu is a great plugin for attaching files documents to Rails models. It’s a rewrite of his acts_as_attachment plugin. While it can handle any kind of file data, most commonly, it is used for attaching images; as a result attachment_fu handles automatic resizing of images, and creation of thumbnails using RMagick, minimagick, or ImageScience.
1 2 3 4 5 6 7 8 9 10
The above will take an image, resize it to 300 pixels wide (automatically adjusting the height to preserve the original images aspect ratio), and to 75 by 75 pixels for a thumbnail, and save resulting images. Combined with a Product model that has_one :image, or has_many :images, and the right form, you can easily manage your product images.
However, an image with both a fixed width, and fixed height, like our thumbnail, can be a problem. If the original, and resized image do not have the same aspect ratio the resized image will be distorted. In this case, if the original is not square, our thumbnail will be look squished in which ever dimension was longer originally. This is not a problem for the main image because we let the height be calculated automatically.
Fortunately, there is a simple trick that allows us to override the method attachment_fu uses to resize image and manipulate it ourselves. Add the following to the ProductImage model:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
and change the thumbnail size to:
Now, if the image size starts with ‘crop: ’, the image will be resized and then cropped to fit. Otherwise, it’s passed on to attachment_fu and handed normally. I’m using the RMagic crop_resized! method, which resize the image using the smaller dimension and then crops the large one to fit. If you are using minimagick, or ImageScience you may need to fiddle a bit with the code. Obviously, you can extend this approach to manipulate the image anyway you see fit. For example you could automatically put a border on the images:
1 2 3 4 5
Or blur them:
1 2 3 4
Or any other weirdness your heart desires. Have fun!
Previously I wrote about how to use public key encryption to automatically encrypt data using Ruby (and thus Rails). Because this method can encrypt data without a password, it’s very useful for securing information received from a form, without the person entering the from having to do anything special. However public key encryption has limits; the amount of data you can encrypt with this method is limited by the key size you use, and, that after a point increasing the key size isn’t a practical an option. We solve this problem using a combination of public key encryption and symmetric-key encryption.
First a little theory; and when I say “a little” I mean it. Under the hood encryption is a headache inducing branch of mathematics. If you want to the literal truth, there is plenty of good reading out there.
Symmetric-key cryptography is what people tend to think of when, and if, they think of encryption. A password is used to encrypt some information, and that same password must be entered to retrieve the information. Under the hood is an algorithm or cipher, which simply put, is a mathematical function that transforms the data into something obscure and then back again. For our purpose we need a block ciphers. A block cipher takes a small, typically 128 or 256 bit, chunk of data and encrypts it. There are many, but in this example we’ll use the Advanced Encryption Standard which is the de facto standard.
(There also exist stream ciphers which work on streams of data, encrypting a phone call for example, but that trade security for speed and can be difficult to use correctly.)
We also need a little glue. Because block ciphers operate on small chunks of data, they need to be applied again and again. However give the same input data the cipher will always produce the same encrypted output; any redundancies in the input will be exposed as redundancies in the output and make in vulnerable to a number of attacks. To avoid this we use a mode of operation called Cipher Block Chaining. CBC using data from one block to further obfuscate the data in the next, effectively hiding any redundancies.
While simple and secure, using symmetric-key cryptography can be problematic; everyone needs to know the password to encrypt data and everyone who has the password (or as the pros say, shared secret key) can decrypt data. This works well if you small set of people who need to know the password and a secure way to distribute it (over drinks in a dark corner of a seedy bar is poetic, if not necessarily secure) but in the case of a web site with hundreds or thousands of people entering data, it’s not practical.
Public-key cryptography can be thought of as Symmetric-key encryption with two passwords, or keys, called a key pair. One, the public key, that encrypts data and another, the private key that decrypts. Because the public key can not be used to decrypt data it encrypts it can be safely given out or installed on a web site, allowing anyone to encrypt data to be sent to the owner of the key. The private key is kept safe and is typically symmetric encrypted with an additional password
The solution is actually quite simple. We generate a random password and use that for the symmetric-key encryption. We then encrypt the random password using the public key, and store both the encrypted password, and encrypted data. When we need to get at the data, we use the private key, and its password to decrypt the random password which, in turn, is used to decrypt the data.
Well, it’s almost that simple. In order to randomize the data, the CBC glue requires a Initialization vector (IV). This article has a good explanation of why, but for our purposes we can just think of it as a second random password we need to encrypt and save.
OK, enough talk, let’s encrypt some text:
1 2 3 4 5 6 7 8 9 10 11 12
At this point we could just save encrypted_data in the database and it would be well protected. So well in fact that we couldn’t get it back. To do that we’re going to need to save the random password and IV.
Generate a key pair. Be sure to choose a good password as this is the one that will decrypt everything.
1 2 3 4 5 6 7
Now extract the public key:
1 2 3
See my previous article for more details on what we’re doing here.
Now we can use the public key to encrypt the random key and IV:
1 2 3 4 5 6
Now if we store all three pieces, encrypted_key, encrypted_iv, and encrypted_data we have successfully encrypted our original data.
Of course we’ll want to get that data back, and to do so we reverse the process:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
password is the password you used when generating the key-pair.
Now let’s put it all together in an Active Record model:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
When creating or updating your model you don’t have to do anything, if “plain_data” is present it will be automatically encrypted. When you want to view the plain text you call “@record.decrypt_sensitive(‘passwd’)”; that could be done with a little AJAX that prompts for a password and populates the “plain_data” field. The encrypted data is only updated when “plain_data” is present. This done so that the record can be updated without decrypting (and re-encrypting) encrypted data (handy in an application were not everyone has access to the sensitive data). To actually clear the encrypted data call “@record.clear_sensitive” and then save.
Setting up the APP_CONFIG hash is left as an exercise for the reader.
Clearly, this screams for a plugin; watch this space.