Tags

mirroring http traffic

11:59am on Mar 09, 2016

Today I needed to figure out how to mirror 100% of inbound HTTP/HTTPS traffic to a second cluster of backend servers, whilst not disrupting the production traffic going to the primary cluster.

There's lots of tools to do things like this, for example duplicator and teeproxy and parallel-proxy. But those are all "code I found on the internet" and possibly questionable, and in some cases, written in node.js, which I don't know if I consider "mature" or "robust" enough for production traffic.

But there's a much better answer. nginx. It's simple:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
server {
    listen 80;

    location @mirror {
        proxy_pass http://mirror.test.com;
    }

   location / {
        proxy_pass http://primary.test.com;
        post_action @mirror;
    }
}

hacker kangaroo

10:17pm on Oct 15, 2014

Building on what Stephen said in his post When is a Martini Not a Martini, I have decided to name a drink.

Some time ago (quite some time indeed), I worked at Federal Express as a box thrower. I mean, package handler. Make sure you pack well, trust me. Anyway, I saw a lot of boxes. At one point, I noticed a box containing Triple Eight Vodka. Being a fan of such things, I was curious, and at some point later I had the chance to try this. Great stuff.

Vodka is one of those things that everyone says when good, shouldn't taste like anything. I think that's wrong - it should have taste, but it shouldn't have bad taste. What's the point of liquor that is tasteless? I'm not an alcoholic, I don't drink to get drunk.

So there was this bottle of Triple Eight in my freezer, and then it was gone. Having recently sampled Crystal Head Vodka, I decided to take advantage of the sale at the local Total Wine and buy a bottle of this as a replacement. And as a bonus, it's even signed by Dan Akryod!

Recently I've been almost exclusively an Old Fashioned drinker. I'm not sure there's a better cocktail (though I do love a good Negroni), and I'm not very picky which whiskey goes into them. But there's this bottle of Vodka in my freezer that I must admit isn't quite as good as the previous bottle, so why not finish it off so I can go back to the Triple Eight?

Today's cocktail is a mashup of the Gibson Martini and the Kangaroo Cocktail - 2.5oz vodka, 0.5oz dry vermouth, and a cocktail onion.

I'm calling it a Hacker Kangaroo.

This is where you get my extremely subtle pop culture reference. Right? ...maybe?

exploited

10:44am on Jul 09, 2014

Fun, I just got hit by this flaw in ElasticSearch.

always handle errors

8:31pm on May 03, 2014

I made this pull request but the author of the library thinks that not bothering to check HTTP status codes is acceptable.

So my code goes from:

1
2
3
request_token, request_token_secret = self.oauth.get_request_token(method="POST")
auth_token = self.oauth.get_access_token(request_token, request_token_secret,method="POST")
self.session = self.oauth.get_session(auth_token)

To:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
from rauth.service import process_token_request
from rauth.utils import parse_utf8_qsl

rsp = self.oauth.get_raw_request_token(method="POST")
rsp.raise_for_status()
request_token, request_token_secret = process_token_request(rsp, parse_utf8_qsl, "oauth_token","oauth_token_secret")

rsp = self.oauth.get_raw_access_token(request_token, request_token_secret, method="POST")
rsp.raise_for_status()
auth_token = process_token_request(rsp, parse_utf8_qsl, "oauth_token", "oauth_token_secret")

self.session = self.oauth.get_session(auth_token)

It's not horrible, but really, why would you ever think it's OK to not handle errors?

bug or feature?

9:25pm on Apr 29, 2014

I've been writing an API for a little project I've been working on for a while, and in searching for a not-horrible way to do OAuth1 authentication, I actually found a Python library that doesn't suck.

Of course, it's not perfect. I noticed today that it doesn't actually handle HTTP error responses - it doesn't even check the return code at all, just assumes that any response it's given will be parseable. Which of course is not at all true in many cases - including in mine.

So of course I've forked it and am working on a fix.

you guessed it - another bug

12:37am on Apr 25, 2014

Found another bug and made a pull request - this time in the 'rauth' library, which does OAuth in a reasonable sane way.

Except for this issue - I still have no idea why they're trying to parse the OAuth response with a utility used for parsing HTTP requests, but hey, I guess if it works for them, fine.

For me though, I need to replace their use of parse_utf8_qsl(s) with json.loads(s.decode()) because my response is proper JSON - shouldn't OAuth responses be JSON anyway?

Whatever, it's late.

EDIT: Okay so it turns out I was doing silly things like not reading the OAuth spec and the response should be a query-string type thing like oauth_token=foo&oauth_token_secret=bar instead, which is what the library parses just fine by default. Reading specs is a good plan, one I encourage everyone to do.

My pull request is still valid though, if you really must break the spec, they have the parser argument already, and it should work in a more sensible way.

yet another bugfix

3:24pm on Apr 10, 2014

Another bugfix for s3cmd.

bugfixin

10:37am on Mar 25, 2014

Bugfix for s3cmd - some issues with command-line arguments not working when I needed them to.

RSS alert feed bot

5:28pm on Mar 11, 2014

Today I created a program to pull data from the RSS feeds our service vendors use for alerts, and either log, email, or instant message (we use Hipchat) to various support groups.

AND, I open sourced it on github. Enjoy!

rom

10:16pm on Feb 27, 2014

More python 3.3 porting, this time an interesting Redis ORM.

EDIT: Changed my mind, someone else has started a 3.3 port which looks like a way better method: mayfield/rom.

[RSS] [atom]
Tags