Lassoing the Clouds: Best Practices on AWS

I recently had the pleasure of speaking at Atlanta PHP and php[tek] 2017 where I presented a new talk, "Lassoing the Clouds: Best Practices on AWS."

As you might be aware, the Amazon Web Services offering is massive.  Between the details of running virtual private servers and AWS's managed services, understanding the best practices when running a production workload can be difficult to grasp.  This talk hopes to point out just a few of the key practices to ensure long-term maintainability and reliability of your infrastructure.

There are a handful of animations in the presentation itself, so I'm posting three versions of the slides below.  Enjoy!

One slide per page

One page per animation transition

One page per animation transition, with presenter notes

On to CrowdTwist

TL;DR: I'm making a move to CrowdTwist, a New York City-based startup providing social and loyalty services for some of the world's biggest brands. I worked with their co-founder and CTO, Mike Montero, from 2001-2005 at Community Connect Inc., where I got my start as a developer in PHP and open source tools. I'm thrilled to be joining him and the CrowdTwist team on what's sure to be an incredible adventure.

Back in 2001, I joined Community Connect Inc. (now Interactive One) as a Senior Network Support Specialist. I was an internal sysadmin, spending my time managing Linux-based file servers, development servers and things of that nature. I had been living in New York just a few short months.

CCI operated what were, at the time, some of the most highly-trafficked social networking sites on the Internet. This was before MySpace and Facebook, of course. BlackPlanet.com was one of the mostly highly-trafficked PHP-based sites on the Internet. We were doing an insane amount of traffic. Our applications had to perform well and scale. We had no choice.

Even as a sysadmin-type, I was surrounded by some incredibly talented developers, who were all working with PHP on Linux and Apache using Oracle. There were caching, using CDNs, and doing things that were still relatively new on the Web. I caught the development bug. I started writing code in my spare time, taking on little projects on the side. How could you not get totally infected in an intense, exciting environment such as this?

In early 2002, I received a call out of the blue from my boss (also our CTO), Mike Montero. On that call, he asked me if I'd be interested in moving to the CCI development team. There was only one way to answer: "YES!"

Thus, in early 2002, just about 10 years ago, I became a developer. The most lowly of the low -- "Associate Software Developer." Over the next three and a half years, I worked my way up to Technical Lead, learning a ridiculous amount from my colleagues. I had amassed this strong set of experience in systems administration and development. I was really growing my skills, and I loved every second of it. I worked many late nights and weekends...and it was an absolute thrill.

Our work was literally being seen by millions of users every day. We were building quality products, all on a home-grown internal framework of sorts. We were doing code reviews. We were writing unit tests. This was how software was built. I never knew any lifestyle but this -- it was my first development gig! This was just the way things were done. This period of time really shaped my personal stance on how to build quality software that was both performant and scalable. I consider myself so very fortunate to have started with this level of experience. It's what gave me such a strong base of experience as a software developer.

In mid-2005, I moved on from CCI and spent almost five years in an interactive agency, Schematic (now Possible Worldwide). During this time, I gained exposure to new, different technologies like Zend Framework and Memcached. This was my first foray into leading major technical projects for clients, but still rolling up my sleeves, diving into architecture and code. I was using my skills from CCI with PHP, sysadmin duties, and databases, and applying them to client work time and time again. I was working in a world that was very different from what I had known at CCI, but bringing so much of that experience forward with me. In mid-2007, we moved to Atlanta, where I stayed with Schematic.

After almost five years at Schematic, I moved on to Yahoo! for a little over a year. This allowed me to get back to my development roots, focusing solely on code and architecture. I had a great time.

In mid-2011, I made a move to Half Off Depot to build an internal development team and grow the technical side of the company as Lead Software Architect. Here, I've was using all of my skills: systems administration, PHP, MySQL administration, managerial duties, recruiting, and working with other departments, such as marketing and design.

Over the past eight months, I've made a huge impact at Half Off Depot in terms of stabilizing the application and its Production environment. I've branched out into using Git and GitHub, Capistrano and Amazon Web Services. I've also had the opportunity to continue sharpening my Objective-C and iOS development skills. Overall, Half Off Depot has challenged me, and I've enjoyed it. I've reaffirmed to myself that I've got a breadth and depth of skills, and that I'm still pretty sharp with all of them. It's also reminded me how much I enjoy a startup environment.

But about a month ago, Mike Montero came calling again -- this time, with an opportunity for me to join CrowdTwist, a New York City-based startup where he's a co-founder and CTO. CrowdTwist is an emerging, unique player in the loyalty space. Think "platform as a service." APIs, user-facing sites, large amounts of data. And an incredible team that's tapping into this data to provide real value for their clients.

When someone you trust and respect comes calling and seeks you out, you listen and explore. And that's exactly what I did. And let me tell you, the CrowdTwist team is INCREDIBLE. I could not be more excited for this career change, both for the opportunity to work with Mike once again, but also to work with all of the brilliant team members and their clients.

I'm in a unique position where I had almost five years of CCI-level experience, coupled with seven years of experience since then. Now I'm going back to work with Mike and the CrowdTwist team, where I'll be able to bring my strong foundation from CCI, along with all that I've learned in the years after CCI. My career has come full circle with respect to the last decade.

I typically like to make a job change, then stay there for at least four years as I did with CCI and Schematic. However, this opportunity with CrowdTwist is so rare that I had to take it. To be with this caliber of talent in such a promising space where they're truly a pioneer? You just don't say "no" to that. Or if you do, you regret it in a few years when they've been wildly successful.

So, on March 12th, I'm joining CrowdTwist full-time. I'll be working remotely from Atlanta, but traveling up to New York City from time to time. I'll be focusing on a mix of back end development and architecture, systems administration, and helping the team continue building quality software.

To my Half Off Depot colleagues, it's been incredible! We've done some great things together. I wish you all the best of luck. Also, this has easily been one of the best team of technologists I've ever worked with. Thanks, guys.

To my future CrowdTwist colleagues, thanks for welcoming me! I'm so thrilled at the opportunity to join you. This is going to be an incredible ride. I'm ready to rock.

See you soon, CrowdTwist! And if you've read this far, thanks. :)

Technical New Year’s Resolutions: 2012

I like to do this every year. It's a reflect and reset-type of thing for me. Here's what I accomplished from last year's resolutions:

I'll keep it short. This year, I want to:

  • Finally release FloodWatch for iPad and Mac. I've already started the static library. This should be done in the first half of the year.
  • Gain a deeper knowledge of PHP 5.3 and 5.4 features, such as namespaces, closures, and traits.
  • Beef up my front end development skills. I've done front end work for months at a time, but never for a long period of time. When I take on this kind of work, I enjoy it, but I want to do more of it in a focused manner. Front end development skills have been my area of weakness for my entire career. I'm a back end kind of guy.
  • Refocus my technical leadership skills. I've led a number of large projects during my career, and I'll get to do more of that in 2012. As a result, I look forward to really sharpening my technical leadership skills with respect to architecture, technical project management, and driving teams of incredibly talented developers.
  • Get back on the conference circuit. For real this time.
  • Blog more. I've started getting back to this recently. I'm going to stick with it.

From where I sit, 2012 is going to be incredible. 2011 was great, but I've got my work cut out for me this year. It's going to be a wild ride. See you around!

What are you going to do in 2012?

Magento for Big Kids: Use a Consistent Cache Key Prefix

This is the first in what I hope to be a multi-part series on using Magento.  I've been working with it since June to run one of our core sites at Half Off Depot. During these six months, we have:

  • Built an incredible team
  • Had our application servers swap to disk and otherwise go belly up, on numerous occasions
  • Moved from FastCGI back to mod_php
  • Tuned Apache a bit to squeeze more out of our application servers
  • Moved from a single front end webserver to multiple
  • Introduced MySQL master-slave replication, when the slave actually in use -- it's not just a hot backup
  • Crafted a respectable Magento application deployment process using Capistrano
  • Cranked out some basic Magento extensions
  • Run countless sales, built mobile apps, launched new web applications and a private API

I don't claim to be a Magento expert after only six months (far from it), but I / we have learned a lot during this time.

In this first post, I'm going to unveil a handy tip:

Always use a consistent Magento cache key prefix.

This can be done in your app/etc/local.xml file like so:

<cache>
    ...
    <prefix>foo_</prefix>
    ...
</cache>

Magento performs poorly out of the box.  The crux of this is that its database is based on an Entity-Attribute-Value (EAV) data model.  Using APC is basically a requirement if you do any reasonable amount of traffic, too.  Big red flags, right? Anyways...

It also uses an aggressive caching model for things like:

  • Front end layout
  • Blocks of HTML used to render pages
  • Magento configuration data (because many core application config options are stored in the database)
  • Translation data
  • Full page cache
  • EAV types and attributes

Because we run on multiple front end web servers, we have to have a centralized cache.  We use Memcached for this. For example, if someone alters a product in our admin tool, which purges the cache item, the refreshed cache item needs to be available to all front end web servers at the same time. An inconsistency between our front end servers is not acceptable. Hence, the centralized cache.

When using Memcached, Magento uses Zend_Cache_Backend_TwoLevels under the hood because it has to maintain a mapping of cache type to the cache keys that make up the data for that type of cache. For example, you should be able to purge the layout or block cache without purging the entire set of cached data.  There's no sort of partial cache purge out of the box possible with Memcached.  You either mark all items invalidated, or delete what you want to, one key at a time. In our case, the mapping of cache type to its keys is stored in our MySQL database. That's not ideal, but...it is what it is for now.

But, here's the catch: Magento / Zend_Cache prefixes all cache keys.  If you don't define a cache key, Magento uses one that is formed with:

  1. Absolute path to your app/etc/ directory
  2. MD5 of this path string
  3. First three characters of the MD5 hash value

This makes all the sense in the world, right?  Maybe it does for some, but it doesn't for us.  We run one web site driven by this single Magento instance.

But, we also have a deployment process centered around Capistrano.  What does this boil down to?

  1. Code releases are kept at /foo/bar/site/releases/[YYYYMMDDHHIISS] of when the code was deployed
  2. /foo/bar/site/current is a symlink to /foo/bar/site/releases/[YYYYMMDDHHIISS] of the most recent release

So, as we deploy code, the absolute path to the app/etc/ directory on disk changes!

Thus, when we deploy a new release, we get extremely high cache churn -- the absolute path to the app/etc/ directory has changed because we move that current symlink!  We basically were rewriting all of our cache with every release, clobbering our Memcached instances and database.

We had to take the site down for a minute or two and refresh the cache in a controlled environment without many concurrent requests taking place. Once the cache had been warmed by a single user, all worked well once again.

Why not, say, throw an exception and barf all over the developer? Or provide a reasonable, consistent default value? Don't change it as filesystem changes occur. That's not intuitive at all. If you're running multiple sites and need a separate cache prefix, you're probably advanced enough that you've considered the implications of your approach already anyway.

So, what's the lesson here?  Define your own Magento cache prefix to ensure that it never changes.

Hope you find this helpful! Here are a few more topics that will be coming soon:

  • Deploying Magento to multiple web servers with Capistrano
  • Centralizing your /media directory
  • Don't touch Core!
  • Abstraction is for wimps: query the database yourself

“Publish or perish”

"'Publish or perish' refers to the pressure to publish work constantly to further or sustain a career in academia." Many years ago, my old boss at Community Connect Inc. (now Interactive One), Michael Montero, was having an article published in the September 2002 issue of Sysadmin Magazine. Around that time before the article was published, Mike enlightened all of us software developers on the "publish or perish" idea -- write articles, speak at conferences, etc. in order to enhance your development career.

I always took that advice to heart. It made sense to me. If you're good at something, you use your communication skills to better yourself and make yourself known. Whether it's writing an article, writing a book, or speaking at a conference, it's important to share your experiences and impart your knowledge to others.

So, that was one of my big goals for the year. I've done the conference thing for a while now, so I wanted to make 2009 the year of writing. By writing, I mean writing code, and writing an article. So, by the time this is posted, my php|architect article, "Make It Mobile with PHP and Open Source Tools" will have been published.

I started it early in the year and finished it in mid-April, many months ahead of the deadline. It's basically my "Rickroll To Go..." conference talk, but in a written form.

It's not some huge triumph of an article...it's not going to win any awards...but it's representative of a goal I set out to accomplish and succeeded at.

So, if you're reading the issue, enjoy! If you're not, go subscribe to php|architect!

Also, special thanks to my php|architect crew for giving me this great opportunity.

I'd like to do some more writing in 2010 (maybe some non-PHP writing, too?), so maybe you'll see me in print again next year. Thanks for reading! Please send along any comments, questions or feedback via email or Twitter.

“Rickroll To Go…” ZendCon Session audio posted!

My ZendCon 2008 talk, “Rickroll To Go With WURFL, PHP, and Other Open Source Tools”, was just released at Zend DevZone as ZendCon Sessions episode #23! If you're just now finding my blog from there, welcome! And thanks to Eli White, Community Relations Manager for Zend, for selecting it for posting.

You can get all of the relevant info using the links below:

Slides and videos of the presentation materials ZendCon Sessions page with audio MP3 audio of the presentation iTunes DevZone podcast

Enjoy, and thanks for listening! Find me on Twitter or email me if you'd like to discuss the materials.

“Rickroll…” goes to print with php|architect!

One of my accomplishments during 2008 was preparing and presenting a new talk, "Rickroll To Go With WURFL, PHP, and Other Open Source Tools". This talk focused on some of the challenges with delivering content to mobile device users, such as limited bandwidth, limited resources on the device, and varying device support for video and audio formats. It illustrated how to use tools such as the imagick extension and FFmpeg to deliver content and an experience that was optimized for mobile devices.

Or, another way to refer to it: if you were ever at a PHP conference and heard of some dude Rickrolling a group during his talk, that was me.

I gave this talk at Atlanta PHP, ZendCon 2008 and PHP Appalachia, and received positive reception and feedback in all cases.

Now, in trying to knock out my New Year's resolutions, I am adapting this talk into an article for php|architect, which will be published around October 2009.

I've only got 3,000 words to work with, so I may not be able to address all three major content areas -- images, video, and audio -- but I'll do what I can. The focus of the issue will be "graphic manipulation," so focusing on the image portion may end up making the most sense given the word limitations. You don't want to read more than 3,000 words worth of my ramblings anyway.

So, keep your eyes open for that issue later in the year! Also, special thanks to Elizabeth Naramore, Beth Tucker Long, and Marco Tabini for the opportunity to grace the pages of their fantastic publication with my words, ideas, and experience.

Also, another special thanks to my co-workers JP Crevoiserat and Joseph Jorgensen who worked with me on a project that inspired the talk and this article. And as always, a special thanks for Schematic for always supporting the community-related efforts of its employees.

Factory Method pattern; ATLPHP tonight!

I'll be giving a "mini-talk" on the Factory Method pattern at Atlanta PHP this evening.

Who doesn't love the Factory Method pattern, right? Good stuff. A link to a PDF of the slides is below. I struggled to come up with a cool example, but an one related to cars was the best that I could do. You'll find some PHP-specific example code in the slides, too.

Grab the slides (PDF)

For further reading on the Factory Method pattern and other classic design patterns, you can always grab a copy of Design Patterns: Elements of Reusable Object-Oriented Software. Enjoy!

Converting lowercase_with_underscores to camelCase

I have a case where I'm retrieving an array of associative arrays from a database as shown below:

Array
(
    [0] => Array
        (
            [id] => 1
            [some_id] => 100001
            [some_company_name] => Foo
            [name] => Bar
            [created] => 2008-12-25 21:13:58
            [last_updated] => 2008-12-30 23:32:43
        )

)

...but I need to represent this structure in XML. I have a pre-defined standard in my XML request/response that the element names are to be camel cased.

Aliasing the field names at the query level is just...a hack. So, I needed a way to convert "foo_bar_baz" to "fooBarBaz" before adding the XML element.

To do this, you can use preg_replace_callback() as shown below, which uses a callback function on every match found.

class FooController
{
    public function barAction()
    {
        $dao = new Some_Dao();
        $rows = $dao->getWhateverRows();

        foreach ($rows as $row) {
            foreach ($row as $key => $value) {
                // $key is "foo_bar_baz"
                $key = preg_replace_callback(
                    '/_(\w)/',
                    array($this, '_convertToCamelCase'),
                    $key);

            // $key is now "fooBarBaz"
            }
        }
    }

    private function _convertToCamelCase(array $matches)
    {
         return ucfirst($matches[1]);
    }
}

There is an e modifier that can be used with preg_replace(), but that requires the replacement string to be a valid string of PHP code. This forces the interpreter to parse the replacement string into PHP on each iteration, which can be quite inefficient. Instead, the preg_replace_callback() uses a callback function, which only needs to be parsed once.

I suppose you could easily do the reverse, though I haven't had a need to write that code yet. :) But...there you go! A little end-of-year tip from myself and the preg_replace() examples.

Zend_Log_Writer_Mail: “Ready for Recommendation”

Almost a year ago, I proposed a really simple Zend Framework module, Zend_Log_Writer_Mail. At its most basic, Zend_Log allows n number of "writers," where a writer is something like writing to a stream, Firebug/FirePHP, etc. At the time of the original proposal, I had done a lot of work with back end batch scripts that emailed developers in the event of any severe errors (warnings, errors, etc.).

Rather than leaving this handy dandy piece of code in Schematic's PHP code library, I abstracted out the email logging piece and proposed Zend_Log_Writer_Mail.

I've received a great deal of feedback, and have only recently committed myself to really pouring over that feedback to polish up the proposal. Thanks to everyone that commented!

So, as of tonight, I polished the code up, made what I felt to be the most worthwhile changes, and marked it as "Ready for Recommendation." Glad I've accomplished something in 2008! Now, what about that iPhone game I was working on? :)

My next Zend Framework endeavor: proposing Zend_Cache_Profiler, ala Zend_Db_Profiler, with my ultimate goal being Zend_Cache_Profiler_Firebug ...something I've sort of written at work, have found really handy, and think that the rest of the community could benefit from.

Maybe it'll be done and "Ready for Recommendation" by 2010. :)