The Ad-Hoc Approach Can Work, Sometimes

Joel Spolsky's introduction to his article How Hard Could It Be?: The Unproven Path at Inc.com states that he has broken seven rules about creating a technology venture. All the rules (except maybe the last one) also pertain to executing software development projects. If I were to draw a conclusion from this article it is this: you can, sometimes, create useful quality software in a reasonable time frame without following established approaches --- i.e. using an ad-hoc approach can work. The article gives one example where this approach succeeded. A few of my own projects and some of my colleagues have also succeeded in spite of breaking most of these rules. I'm welling to bet a great percentage of the single person or small team open source projects have also been successful using the same approach.

Don't get me wrong, I'm not advocating the ad-hoc approach for software development projects. In fact, even suggesting so leaves a bad taste in my mouth and gives me a sick belly. But I have to be realistic, for certain types of projects, it does work. I believe the success of using this approach depends heavily on the project team and the nature of the project. Off the top of my head, here are a few ingredients the project should have to use the ad-hoc approach successfully.

• The team must be small. The larger the team the less chance the ad-hoc approach will be successful.
• The entire team must consist of excellent developers: smart, experienced, knowledgeable, passionate (Joel states this in a slightly different way at the end of his article).
• The team must have a deep, intrinsic understanding of the requirements for the software.
• The team must believe in and take ownership of the project.
• The entire project should be built in-house --- minimum amount of collaboration with and dependencies on external teams, especially those from other companies.
• The software is being built from the ground up.
• The product is not an "enterprise solution" built in an IT shop where
  • lots of red tape, bureaucracy, hierarchy, turf wars, and office politics can get in the way; and,
  • the software being built is not tightly coupled with lots of other enterprise software.

With these points in mind, lets revisit Joel's seven broken rules (sometimes liberally paraphrased) and try to identify when and why these rules can be broken without negatively effecting the project. (The following comments are opinions based on my experiences, no research on their validity has been conducted.)

Hire only the best programmers. I whole heartedly agree with this rule for all projects: it can't be broken without consequences. Joel thinks he has broken this rule by hiring Jeff Atwood and Jeff's team of developers without checking if they "could write good code". I don't think he really broke this rule at all since Jeff's reputation has being an intelligent, passionate, good developer is generally accepted by the development community. There is very little risk in hiring a developer as well respected as Jeff.

All developers and management staff must be in the same office. When developing software before mid 90's, this rule could not be broken. This is no longer the case, especially for projects with a small number of developers. All team members can effectively and efficiently communicate, perhaps more so, using readily available communication and collaboration technology, including the basics such as email, chat, video chat, twitter, wikis, phone, video phone, and video conferencing. As an example, consider 37 Signals, a company that embraces the distributed model in addition to building tools to help support it.

Plan before proceeding. Planning is always done, but when, how much, how formal, by who, and if it's documented or not can change for each project. For projects that meet most of the criteria outlined above, it is sometimes OK to do minimalistic up front planning, as opposed to complete project plans where the entire project is broken down in many fined grained tasks and extensively documented. The project plan can occur in the team leaders heads, perhaps with some notes so things are not forgotten (but with minimum detailed formal documentation), and keep at a very high level. The plan, written down or not, is given just enough detail to get the project going and keep it going from one major task (short phase) to the next. The plan is iteratively fined tuned and changed as the project proceeds. Task level planning also occurs as developers plan how best to tackle the next new task. Again, this is very informal and may not be documented.

Issue tracking. For new projects you can probably make do without a formalized tracking system, mainly because most issues (e.g. bugs) will be identified after the project is done and the product is in use. Nevertheless, I would recommend some place where developers can note potential improvements and known unresolved issues they encounter during development. In the most basic form, these items can reside in code comments flagged with tags like TODO (which can be automatically extracted into to TODO file using a simple script), or in a simple TODO.txt file at the root of the project. For long term products with on going maintenance and feature development, and where many different developers have to work on the product, issue tracking is a must. A sophisticated system may not be necessary, but ideas, bugs, issues, features, and tasks need a centralized place to live so

• issues are not forgotten as different developers work on the product over time;
• a place exists to record discussions and information gained about the issues;
• issues can be organized (in particular they can be prioritized);
• issues can be effectively communicated to all team members;
• responsibility for issues can be assigned, tracked, and communicated; and
• it help managers track issues and their status, progress on addressing them, and in planning for new phases of development.

Test software before releasing it. Unless you have perfect developers, I don't think this rule can be broken. What changes is who does the testing, how formalized it is, the types of testing performed, and when it is done. I would argue that for projects with really good, experienced developers, you can probably minimize the amount of formalized testing since (1) the bug per line of code introduced during construction will be low (compared to inexperienced, bad developers), and (2) these developers will do significant testing as they develop. For small or new projects that developers are passionate about, they will also do the end user testing themselves: no formalized testing teams necessary. In some cases, it may be appropriate to release the software in alpha and beta states to get user testing in the field, further reducing the need for formalized testing and test teams. In many cases, such as mission critical software and software which must interface to existing system (especially legacy ones), this ad-hoc approach may not be sufficient nor efficient.

Create schedules to ensure a project is delivered on time, within budget, and meets the requirements. For some projects, especially new projects where the software being built is charting unexplored territory, detailed schedules may not be necessary or of little use because the requirements and/or the tasks that need to be completed and the complexity of them is not well understood (See Frequently Forgotten Fundamental Facts about Software Engineering). Estimates are generally so poor that they are often no better than a general all encompassing guess anyway. Alternatively, the project might be small, will understood, and the developers are very experienced in building similar software, so developers know exactly what needs to be done and can give an accurate encompassing estimate such as "6 to 8 weeks" thus reducing the need for detailed schedules.

Decide on how to make a profit before building the product and measure its success by profit. This rule is about the business side of software development, so I won't comment on when and why you can break it. I will state that a project can be classified successful without making a profit. For example, it is useful to someone, it has quality, and it meets its requirements. Many open source projects are definitely successful, but they make no profit. The point Joel makes about building useful software first, then worry about making profit from it later, reminds me a lot of Steve Yegge's article Business Requirements are Bullshit.

References

1. Joel Spolsky, How Hard Could It Be?: The Unproven Path, Inc.com, 2008.
2. Steve Yegge, Business Requirements are Bullshit, 2008.
3. Robert Glass, Frequently Forgotten Fundamental Facts about Software Engineering, IEEE Computer Society, 2008.

RIAs and the Future of the Open Web

In The Struggle for the Soul of the Web Chris Keene writes:

Just because the web has been open so far doesn't mean that it will stay that way. Flash and Silverlight, arguably the two market-leading technology toolkits for rich media applications are not open. Make no mistake - Microsoft and Adobe aim to have their proprietary plug-ins, aka pseudo-browsers, become the rendering engines for the next generation of the Web.

and

The worse the underlying browser is at rendering rich widgets and media, the more developers and users will want your plug-in. If you are both the vendor of a browser (say IE) as well as the proponent of a plug-in (say Silverlight), then the incentives get truly twisted.

I believe this is why Microsoft (MS) has hijacked JavaScript 2.0 (I can't figure out why Yahoo didn't support it) and is hijacking HTML 5. These standards would enable other companies to more easily develop rich widgets and media frameworks that would compete directly with Siliverlight. From MS's point of view, that is unacceptable. Although both Adobe and MS have "pseudo-browsers", MS is a much bigger threat to an open and standard internet because they also create one of the worlds leading browsers, Internet Explorer (IE) --- the potential for proprietary lock-in here is huge.

Consider what happens with increasing adoption of Silverlight. First, Silverlight no longer works so well in non-IE browsers, then a year or two later it's only supported by the top two browsers, and once a critical mass is achieved, it only works in IE. I can see the website banners now: "viewable only using IE and Silverlight". What browser do you think most people will be forced to use? And since IE is tied to Windows, websites will essentially become Windows OS dependent. Moreover, developers will be forced to use the .NET framework for Silverlight, that leads to developers having to use MS development tools on a MS OS... where does it end? Why would MS even care about or need open web standards once all users are locked into using IE+Silverlight+Windows?

I think Silverlight --- the VM and its development tools --- is a great solution for RIAs. However, I'm a strong supporter of open standards that benefit all of our industry. As such, I will avoid, as much as possible, Silverlight for fear I may contribute to the demise of our open web.

References

[1] Chris Keene, The Struggle for the Soul of the Web, AjaxWorld, ajax.sys-con.com, Dec, 2008.

Great Developers Need Great Office Space

As written by Jeff Atwood for Stack Overflow podcast 31 [1]:

Joel [Spolsky] justifies having a nice office space as 1) a recruiting tool 2) enabling higher programmer productivity and 3) the cost of a nice office space is a tiny number relative to all your other expenses running a company. I [Jeff] argue that companies which don't intuitively understand why nice office space is important to their employees who spend 8+ hours every day there... well, those companies aren't smart enough to survive anyway.

Enough said.

[1] Podcast #31, Stack Overflow Blog, blog.stackoverflow.com, 2008

Methods for Speeding Up Your Website

YAHOO Developer Network has an excellent article (Best Practices for Speeding Up Your Web Site) that describes 34 ways to improve your website performance. You can also see live examples of 14 of these items at 14 Rules for Faster-Loading Web Sites.  The article is especially pertinent to developers who make heavy use of JavaScript+AJAX, HTML, and CSS (as opposed to Flash, for example) in their websites.

Following is a listing for easy reference: Make Fewer HTTP Requests; examples here Use a Content Delivery Network; examples here Add an Expires or a Cache-Control Header; examples here Gzip Components; examples here Put Stylesheets at the Top; examples here Put Scripts at the Bottom; examples here Avoid CSS Expressions; examples here Make JavaScript and CSS External; examples here  Reduce DNS Lookups Minify JavaScript and CSS; examples here Avoid Redirects; examples here Remove Duplicate Scripts; examples here Configure ETags Make Ajax Cacheable Flush the Buffer Early Use GET for AJAX Requests Post-load Components Preload Components Reduce the Number of DOM Elements Split Components Across Domains Minimize the Number of iframes No 404s Reduce Cookie Size Use Cookie-free Domains for Components Minimize DOM Access Develop Smart Event Handlers Choose <link> over @import Avoid Filters Optimize Images Optimize CSS Sprites Don't Scale Images in HTML Make favicon.ico Small and Cacheable Keep Components under 25K Pack Components into a Multipart Document

Why Use Abobe Flex?

Here are some reasons you might want to use Abobe Flex (Flash) to build the UI (or some part thereof) for your next web application:

• It's an open source development kit.
• It comes with everything needed to build and deploy Flex UIs. It does not include an IDE, but you can buy an IDE or plug-in for Eclipse to dramatically ease development.
• It is supported and principally developed by Adobe. Hence, it has commercial backing with invested cooperate interest. Some would argue this lowers the risk of Flex becoming vaporware anytime soon. Adobe also offers commercial support.
• It has a large and growing development community. For example, it has a sister open source project for unit testing called FlexUnit.
• It is used by many large and small corporations.
• It is based on standards (at least we can argue it is; JavaScript 2.0, HTTP, XML, etc.) and proven technology, such as Flash.
• It has extensive documentation.
• It has a large library of existing UI and non-UI components. One can also buy libraries of custom components (e.g. charting and graphing).
• It is flexible and extensible. For example,
  • it can be integrated into any existing web page without completely taking over that web page (it can do that too if desired!);
  • all GUI and non-GUI components can be extended and customized;
  • it can access a number of back end data sources and application frameworks, including Java Servlets, Flex Data Services, and REST; and
  • it can use different communication mechanisms, including raw TCP/IP sockets, HTTP, and SOAP.
• It works well with Java back ends, such as servlets and EJBs. Hence, you can harness existing expertise and code base with few changes.
• The development model is easy to use and understand. For example, you can use XML to layout GUI components, JavaScript 2.0 (which is Java like, at least more so than JavaScript 1.5) for attaching behavior to the components, and a simple set of APIs for accessing the server.

Although this is supposed to be a post about why you should use Flex, I have to add this negative point because it caused me no end of grief in my last project: Flex/Flash does not have a component to display standards compliant HTML/CSS. So, if you're like me, and you have some existing HTML content you want to display in the UI, you're out of luck. There are some options available, such as rendering content outside Flash and overlaying it on Flash using IFrames, but I was never satisfied with them. Here are some links about using such approaches:

• Drumbeat Insight Component
• Embedding HTML in a Flex application using an IFrame
• Don't Use IFrames for HTML in Flex

Conditional CSS

Even though CSS is a standard, there are differences in how web browsers render and support CSS. By far the biggest deviant is IE6, which has the poorest compliance of the major browsers I support (FireFox, IE, Safari). In some cases you can simply tolerate the differences, but more often than not you have to find workarounds to convince the non-compliant browser to do what you want. I generally target FireFox first to get what I want (because it has good standard compliance and has FireBug) and then tweak the CSS to workaround IE issues. Following are two methods to conditionally include CSS depending on the target browser. CSS Hacks The first method is to exploit CSS parsing bugs in browsers (mainly IE) to accept or ignore CSS attributes. This is often called CSS hacking. Although commonly used, it is not recommended [1,2]. Two common hacks prefix CSS attributes with a special character to select browser and browser version: *attribute -- for IE 7 and below (I haven't tested on IE 8) _attribute -- only IE 6 and below attribute -- all other browsers Example: div { *width: 20px; _width: 20px; } Conditional Comments The second method is to use conditional comments [1,2]. This relies on a feature in IE to conditionally include HTML and CSS content using special commands embedded in comments. This is the recommend method because it does not depend on bugs to work. However, it is not as tidy or simple to use as CSS hacks and it can't be used for other browsers. Furthermore, it relies on modifying the HTML source to work. The following example includes the CSS resource file main.css if the browser is IE 6. <!--[if IE 6]> <link rel="stylesheet" type="text/css" href="main.css" /> <![end if]--> You can also use comparators such as lt and gt. This example only includes the CSS file if the browser is IE 7 or less. <!--[if lte IE 7]> <link rel="stylesheet" type="text/css" href="main.css" /> <![end if]--> References [1] http://www.javascriptkit.com/dhtmltutors/csshacks.shtml [2] http://www.quirksmode.org/css/condcom.html