SuperSatellite

dotCMS 1.9 Upgrade Preview

// October 23rd, 2009 // 8 Comments » // Software, Tech, Web

The upcoming dotCMS 1.9 update is still at least about a couple months away from release (give or take), but the current SVN trunk code is already starting to shape up with a lot of the upcoming enhancements.  After testing out some of the new code, I thought it’d be worthwhile to share some screenshots and descriptions of what to expect, because it’s not a little change by any means.  The current release of dotCMS is 1.7a, with a 1.7b maintenance patch currently on the roadmap sometime soon.  1.8 is not being skipped, but instead, because of the number of changes that were slated for 1.8 and how they tied to 1.9, they’ve just rolled the whole series of updates into one big 1.9 upgrade rather than have two important updates come close together.  1.9 is likely to be the final big update before 2.0 (at least according to JIRA currently).  So, the future currently looks like 1.7a -> 1.7b -> 1.9 -> 2.0.

If you want to try out what I’m describing below, the code is available in SVN (linked above, username and password both dotCMSUser).  I recommend using it with a PostgreSQL database, as that’s what they develop on, and other databases might not be fully tested yet.  I want to serve up an initial warning too, that this code is absolutely not ready for a live environment.  I don’t mean that from a casual, you might hit an esoteric bug here and there way – I mean it in a you can’t run a web site on this code yet way.  There are a number of features coming that I couldn’t preview yet (like binary content fields in structures and some permissions options), because they flat out aren’t working yet.  I hit a number of areas with those kinds of problems, so keep that in mind if you want to try it out; still very alpha and still very under construction.  But, it doesn’t hurt to familiarize yourself with upcoming changes, since if you’re like me and running a huge site in it, it can be helpful for planning your upgrade path.

One of the big, less visible changes was moving to the Dojo framework for all the Javascript stuff done in the back end.  This is readily visible when you go to log in, as the login block is now a modal window, and you get a nice login progress bar as the back end loads.  This is just shouting for the addition of Google Gears support (a  la WordPress. I actually started work trying to build in Gears to an earlier release, but had trouble with the amount of Javascript used in the back end).  Even lacking Gears, the back end is noticeably faster and more responsive to the user.  This may have other repercussions if you’re a heavy Scriptaculous user, but I don’t know yet, so just be aware that they are moving on to a different framework for the future.  But more to the point…

…holy crap it’s different!  Indeed, the whole admin interface has been redesigned and also made i18n compliant (that should make you international folks really happy).  This is similar to the change that took place when the page preview area was redesigned for 1.6.5.  Navigation wise, everything is still very much the same, so you don’t have to anticipate relearning everything luckily.  Basically it’s just got a new skin over the top to make everything look nicer and navigate better.  But, if you have a lot of lower level users, you’ll certainly want to brace them for the change, since just looking different might confuse them, even if stuff is relatively in the same place.

The content search page is a good example of the updated look, but it’s still very much familiar when compared to the old layout.  Same search boxes, same basic results table.  Some things like the “Show Query” button have moved to a more useful location.

So,  you can add content the same old way, and the familiar TinyMCE interface is there.  But gone are the Text and WYSIWYG tabs at the bottom edge of the field, replaced by a WYSIWYG toggle button.  Click it.  Do it.  I DARE you.  Yes, that is just what you think it is, the old, plain “text” mode has been updated to include a syntax highlighter and line numbers.  I think I’m crying right now, I’m so happy.  This definitely ramps up the usefulness of the text editor several degrees, and will make editing VTL files so much nicer.  Hopefully this is implemented everywhere you have code editing fields.

One of the most substantial changes was to the permissions system.  Gone are groups.  Instead, roles now control everything, and they are hierarchical.  The role manager has been completely updated to reflect the changes, and allows you to list users by role (yes!).  You’ll also see improvements to how permissions work related to different parts of the system.  The 3 factor (view/modify/publish) permissions have been dropped in favor of task specific permissions, and also user specific permissions as well.  One thing that I’ll be interested to see is how the upgrade process handles converting groups into roles and maintaining things coming from 1.7a sites, especially large, complex ones.  I envision this being an opportunity though to redo permissions in a much more efficient way.  It’ll be a lot of work I expect, but well worth it.  Be sure you keep this change in mind when upgrading, since it will have probably the most profound and widespread impacts to how you work in the CMS (Content Management System).

One of the other big changes is the addition of a form manager.  Instead of the complicated, inflexible form submission process used currently, a new structure type has been added that combines with front end content submission.  That makes three different structure types (content, widgets, forms) to choose from now, and greatly enhances the collection of data from users to repurpose elsewhere.  Now you also can leverage all the power of structures for data collection.  Like permissions, converting to the new system will involve a lot of work, especially if you have a lot of forms, but it will likely be well worth it, since it makes nearly every process of form use easier and more powerful.

That’s your initial look at the system.  I’ll do a more thorough review once the code gets closer to a beta release and is more functional.  For now though, I think it’s clear that this coming update is one of the biggest, if not the biggest update that we’ve seen for dotCMS to date.

A Day with dotCMS 1.6.5

// December 2nd, 2008 // 5 Comments » // Software, Tech, Web

It was long coming and well worth waiting for.  On Halloween dotCMS released version 1.6.5, and feeling confident in the code base release, we finally tested and rolled out the upgrade on our production servers (I try not to jump on updates right away, just in case there are any surprise bugs that need to be worked out).  Naturally, we tested things out on our dev box first, were satisfied, and just before Thanksgiving we finalized the upgrade on our live server.  I can say with confidence that this upgrade has proven to be a nice kick in the butt, without a single hitch so far.  Even the improved edit mode interface hasn’t drawn any complaints for being “different.”

A peak at the new dotCMS 1.6.5 edit mode

I’m among the people who think that 1.6.5 is really more like a 1.7 in that it’s not just a minor patch of the 1.6 codebase – there’s a lot of new crap.  Among things to look for:

• WebDav is all new
• Edit Mode is all new
• Widgets have been added
• Updated TinyMCE
• Caching is now in memory instead of disk
• Lucene index has been moved
• PDF page output macro
• A whole crap ton of other stuff

All in all, it’s like 322 different changes, fixes, and additions. Nothing to sneeze at.  And it all matters.  First off, compared to 1.6.0.10, it’s substantially faster on both the front and the back end thanks to the improved caching.  And it flat out blows away 1.5, so if you’re still lingering on that version, I suggest an upgrade pronto.  The speed is coupled with a less noisy log as well, making it easier to tell actual errors from random chatter from the system.  They went through and completely refactored the WebDav interface, so if you use it for bulk adding files or live editing, you should notice a substantial boost in speed there, as well as the ability to autopublish files uploaded through it (great for photos for photo galleries, or live editing things like CSS (Cascading Style Sheets) or VTL files).  The upgrade process itself didn’t feel overly painful either.  One thing that has been a constant complaint was that the system was a little overly painful to actually upgrade, especially from much older revisions.  What I can say is that going from 1.6.0.10 was about the fastest and smoothest upgrade I’ve done so far (that wasn’t just a simple maintenance patch).  SVN update, copy back some server settings, ant clean and deploy, reindex, and we were pretty well off and running (on top of that, I was also switching SVN branches as well).  I could have probably even gotten away with using my old Lucene index, but, fresh system seemed to call for a fresh index.  Interesting side note, I was talking with Chris Falzone over at Edinboro, and he thought upgrading from 1.5 straight to 1.6.5 proved easier than going from 1.5 to 1.6.

My apologies for no real stats on this on my end, but prior to rolling out the upgrade live, I did do some load testing against our 1.6.5 test server using Apache JMeter, and the results were excellent.  I was able to run up about 100 threads within a 30 second time period without drops.  Actually, the machine I was testing with crapped out before the server in some cases.  I also saw some stats that Jason, their lead developer, shared on the mailing list after testing it with Siege that I’ll pass on:

I wanted to share some performance numbers with you.

The test was run on a server with 8 GIGs Ram Dual Quad-Core Xeon E5320 / 1.86 GHz processor running the latest Ubuntu LTS. The server had Postgres 8.3 install locally and Java 5 64bit.

dotCMS configuration
JVM 1536M of memory for the start and max of the heap. The connector was set to 380 Max thread and the Db had 390 max db connections.

The benchmark tool used was siege. We ran siege on 2 separate boxes each with 180 concurrent connections and 0 delay between the connections, so the traffic was a full 360 users attempting to hit the server throughout the entire test. For those who use siege this was the command siege -f urls.txt -r 100 -c 180 -b -i. I will attach the urls used for the test. They were a mix of urls from a starter site. The test was of course performed on our lan. There were 3 switches between the server and the machines running siege.

The total test results were
Total Transactions : 35987
Availability : 99.97%
Average Response time : 0.85 seconds
Total Concurrency : 313.12

Here are the individual numbers from each siege machine.
Transactions: 17996 hits
Availability: 99.98 %
Elapsed time: 97.50 secs
Data transferred: 145.27 MB
Response time: 0.85 secs
Transaction rate: 184.57 trans/sec
Throughput: 1.49 MB/sec
Concurrency: 157.58
Successful transactions: 17996
Failed transactions: 4
Longest transaction: 28.39
Shortest transaction: 0.00

Transactions: 17991 hits
Availability: 99.95 %
Elapsed time: 97.65 secs
Data transferred: 145.22 MB
Response time: 0.84 secs
Transaction rate: 184.24 trans/sec
Throughput: 1.49 MB/sec
Concurrency: 155.54
Successful transactions: 17991
Failed transactions: 9
Longest transaction: 29.46
Shortest transaction: 0.00

We also have some similar numbers with 1.6.5 running on VMWare ESX Server. We will be sharing them soon here. Need to gather all the compiled data for it and specs for the test.

On of the two big, important updates you’ll see a lot of is the much improved edit mode interface, seen above.  Gone are the clunky, vertically columned edit, preview, and live mode buttons.  They’ve moved to the top, and been replaced by a clickable side bar that lets you hide the whole side console completely.  Some much needed UI facelifting was applied with nicer icons and a splash of color.  On the page, container interfaces are squared up nicer, and dropdowns were (mostly) eliminated in favor of one click editing, sorting, and deleting.  That doesn’t sound like much, but if you did much with Flash, you probably fell prey to the dreaded layer monster, where the popups would pop “under” the Flash interface, making it hard or impossible to click the options there.  All in all, the changes were largely cosmetic, somewhat functional, and much needed.  It just feels a little more put together now.

Example of the Widget Modal Window

The other big change is somewhat related, and that’s the addition of widgets.  Widgets have been discussed before, usually as pieces of Web Page Content that are just designed to be dynamic.  This is all well and good, but it didn’t really allow for proper segmentation of functional elements of a site.  Now, widgets are properly defined with a structure.  You set them up like structured content, except your fields basically serve as the variables for a macro.  Think of it as a UI for macros.  The primary difference being a macro can be embedded in content, while a widget goes in place of content.  Why this is so awesome is because one, like I said, it allows you to properly segment dynamic tools for your site, and two, even though a macro is only like a line of code, we all know that for some people one line is just one line too many.  This let’s them fill out a simple, non-threatening form, and see results.  The interface itself is cool, as adding a widget brings up a nice little modal window with all the widgets defined available right there in the page.  Plus, any container can be widget enabled, so that you can make containers fairly multipurpose, in the event that you want one container on a particular page to do something different, but still adhere to a basic layout option.

Speaking of the modal window, I would love to see this applied throughout the site to get rid of popups all together.  Get rid of the content search popup forms for reusing content. Use a modal window for quick adding or editing of content.  It makes people feel connected when they don’t physically have to juggle windows, especially when in some cases (like adding a picture to content), the popup adds information to a different popup below it, and it doesn’t go away when you click it, so you don’t actually get any feedback from the system that a change has taken place.  So there’s a wishlist item from me.

What makes this so worthwhile is that there is a definite maturing of the code base taking place.  The attention to the UI of edit mode, while not changing substantially in a functional way so much, shows dotCMS becoming a much more professional system that cares as much about the user experience as the functionality.  This is an excellent sign as we get ever so closer to what should be a real dam breaker of a version 2.0 next year (hopefully), and the rate of progression is fairly steady.  Like I said, rather than being 1.6.5, this would really almost qualify as a 1.7 in my books, but hey, I don’t do the versioning.

RIP Drupal, you lose.

// May 21st, 2008 // 7 Comments » // Scripts, Web, Wordpress

After about a year, I am kicking Drupal to the curb.  I fully recognize it is a robust, capable CMS (Content Management System).  So are a lot of other systems.  But it’s just not fitting me well.  Sometimes I think CMS usage is as much about tastes as it is strict functionality (actually, I’m sure of it).  Sure, a sweater might be warm, but if it’s ugly (with a big clown on the front), you might wear something a little lighter for the sake of having something more appealing (and less clowny).  That’s where I am at.  I’m tired of clowns and sweaters.

I originally adopted Drupal to run Penpedia when I launched the site for a few reasons.  One, about that time I had started getting into Drupal just through the course of investigating different CMS’s.  I did like that user accounts could have their fields customized.  And I found a module that allowed for hand in hand single sign on authentication with the other half of the site, which ran MediaWiki.  That also made it appealing.  The longer I used it though, the more the rough edges started showing.

First, I didn’t want to start writing templates for a new system.  This is in part because I already know how to template other systems like WordPress, e107, dotCMS, and a couple others.  As a result, I start losing patience for learning others.  Not a huge deal, because theming Drupal isn’t terribly hard, it was just that I didn’t want to.  The theme I ultimately grabbed was only marginal, but I just never got motivated to write a custom one.  And frankly, I’m a little ashamed of that, because the design of the site is not indicative of my abilities.  But what did drive me nuts was the lack of any WYSIWYG editor built in.  And the module that enabled the feature was disgustingly complicated, and caused a lot of clashing with the code stripper.  The added steps of permissions and profiles for it to work right was just way beyond necessary.  Which leads me to the next thing.  The permission system didn’t please me.  It didn’t work how I’d expect, and seemed far to complicated for what it was doing.  That’s what I love about dotCMS, you can’t beat their permissioning system.  Doing complex, and sometimes even more routine, tasks in the back end of Drupal generally felt like a power struggle between me and the code, and I loathed the idea of having to go in and tweak anything.  And just to round it out, I’ve never liked the “node” concept.

But, just to remain clear, if it works for you, then great.  It just hasn’t meshed well with me, and has actually discouraged me from developing the site better.  That’s why I plan on changing things over there to WordPress later this week.  The MediaWiki portion will remain unchanged, though there might be some tweaks to login stuff, as I am investigating what needs to happen to maintain single sign on with the two.  Plus, between here and there, I’ll only have one software package that I’ll need to worry about now, as opposed to two, which should encourage more development and activity on my part on the Penpedia site.  I think this will be a move all for the better, and it should benefit the site well.

The moral of the story?  There is a lot of value to standardizing on a CMS, and sticking with what you know.  Even if a system isn’t perfect, if you are familiar with it and know its capabilities, I think that beats out picking a robust system that just doesn’t click for you.

dotCMS: An Introduction

// January 29th, 2008 // 14 Comments » // Scripts, Software, Tech, Web

I felt it was time to take a closer look at a content management system I am getting very involved with. One might say I’m drowning in it. I prefer to think of it as being choked by it, heh. Really, it’s a very interesting system, and a fantastic tool for the price (free!). I have worked with a number of CMS (Content Management System)’s over the years: Coranto, e107, Joomla, Mambo, WordPress, Drupal, and a few others that I’m sure I’ve forgotten about. It’s easy to get locked into one or two for various purposes, which had been true with me for a long time. The problem is, my short list there is only good for small to mid range sites. Now I’m webmaster for a site of better than 10,000 pages and 70+ editors. Those open source solutions don’t easily or always comfortably scale up to those needs.

That’s where enterprise grade content management comes into play. Swinging big price tags, naturally. You start learning names like RedDot, OmniUpdate, Hannon Hill Cascade Server, Serena Collage (discontinued), and Ektron cms400.net. You also learn they don’t come cheap, some breaking the $80,000 mark. $15,000 in this group is cheap. In some cases, you don’t even have to host it. But the support, features, and scalability you get from this class of tools is in a different league. Wouldn’t it be nice if you could get the best of both worlds though? Something like a Typo3 or Zope/Plone system, but with enterprise grade support and a learning curve within reach of your users? A tool that was open source to boot? Generally there just isn’t demand for this sort of software, at least not enough to make a successful project, it’s enterprise grade and enterprise cost because it’s an enterprise market. However dotMarketing has stepped up to the plate. They have developed dotCMS to augment their design and consulting business, but released it free, charging only for support.

I need to be fair though. dotMarketing really stood on the shoulders of the work done at the Liferay Portal project, that’s where it started. Liferay is the underlying framework that drives dotCMS, written in Java, and running on top of Apache Tomcat. But they forked the project and went a long way towards customizing everything you see and interact with. It isn’t just a fancy admin theme dropped over someone else’s work. It is also tied together intimately with Apache Velocity, which is very slick for writing dynamic templates. Needless to say, this isn’t a simple one-off , self contained type of application. It’s build on several different technologies, all open source, to try and provide the best of each tool to the end user. When you download it, everything is packaged up that you need though, you don’t have to hunt anything down separately.

This does make it more complex than other open source CMS options. Systems like WordPress or Drupal can be installed to any server that runs PHP and a flavor of database server. On the other hand, dotCMS requires you to have root access to the server to install things like the Apache Tomcat server. This requirement has hurt dotCMS’s market penetration, as many web site owners do not have that kind of server access, opting instead for cheaper hosting solutions. That is a shame, because dotCMS is easily one of the most powerful open source options available, with a much shallower learning curve than equally flexible solutions (I’m looking at you Typo3). It’s also pretty resource intensive, so you won’t be dropping this onto a lightweight web server you built five years ago on an old AMD K6-2 system you had lying around. Plan on needed a couple gigs of RAM and a modern processor to heft it. You can run it in a dev environment in a PC okay, but not with less than a gig of RAM (I know, I tried, I failed). It might be open source, but it is truly enterprise grade, and is built for an environment that reflects such.

Part of what makes dotCMS so good is the way they approach content. As an administrator, you can create structures. These structures result in a form-like input system, that allows you to pull out some or all data for a type (say, a news posting, or product review). Any structure can easily be attached to an RSS feed. Each structure is unique, but can be dynamically related to one or other other types of content in other structures. For instance, you have a structure for “blog posts” and one for “comments,” and the comments can be related to blog posts. But you could also relate certain comments to articles, or news postings as well. This is a basic CMS feature used in most systems, but with dotCMS you see exactly how it works, and can modify or expand it. Better yet, create your own. Players related to teams related to sports. The flexibility allows you to develop to your heart’s content. And you don’t have to be a programmer to do it. Admittedly, relationships aren’t super easy to understand out of the gates, but an example or two gets you going. They really got things right with regards to how content should be created and used, and did a good job taking the cork out of the bottle for the end user to go nuts with it.

Speaking of examples, they do have a complete demo site running that allows you to both test things, and also see how certain things are accomplished. This can be very useful if you want to duplicate some kind of functionality, but need an example. They also outline plenty of macros that allow you to easily accomplish common tasks, like creating slideshows, navigation, MP3 players, RSS feeds, and more. One thing their demo site doesn’t showcase is that it can even run multiple domains through one back end, so you could be running demo.site.com, and blogs.site.com, and www.site2.com all through that one instance of dotCMS.

If you would have asked me before, I probably would have said that Drupal or Typo3 had the best permissioning systems available in open source. That’s no longer even remotely true. dotCMS has a fantastic permissioning system that allows granular access to both content and admin areas. Groups are used to define what a user has access to at the back end, while roles determine what they can use, see, and access with respect to content. This combines with a workflow system that can be used to make sure content is properly reviewed and approved before it hits the page. It’s simple, and it works. Well. On the user side, you can easily tie into an LDAP or Active Directory environment, though it requires manual set up. Users can be tracked through the system to provide some basic analytical data, and attached to CRM functionality to label, organize, and communicate with them. This goes far beyond any other competitive product in open source.

Where it’s lacking: documentation. Right now, it’s pretty scattered and different documents can be out of sync with others. I understand this is changing, and that they are moving to a system a la Apache’s documentation method. It is definitely needed. Luckily, their mailing list is pretty active, and they tend to be responsive on bug reports. I think it would be nice to see a little more stuff built in by default. For instance, dotCMS can be a blog, or host them, but you have to build the structures and relationships yourself. Tools like that are common enough that it wouldn’t hurt to just have it already set up. There is also no file structure, in the normal web sense, so no FTP (File Transfer Protocol) type functionality, you have to upload through the system or via WebDav, if your server supports it.

I will be at the dotCMS Open Minds Conference next week in Miami. After that (and maybe even during), I’ll be following up with some more detailed information regarding their CMS. I’ll get a bit more into how certain things actually work, and what we’re doing to make it work to our advantage. In the mean time, their portfolio can give you an idea of what the system is capable of.

Update (08.04.21): If you are looking for additional support, be sure to either check out the dotCMS mailing list, or visit our newly established forum community. Also, expect new and improved documentation and support tools from the dotCMS development team around the 3Q of 2008.

Don’t Contribute

// January 24th, 2008 // 6 Comments » // Software, Tech, Web

What follows is a slightly re-edited (for clarity) version of my thoughts on using Adobe Contribute to run a site. It was originally posted to the uwebd mailing list during a discussion of different CMS (Content Management System) options that are out there. This was in response to a question directed to me regarding what I considered a “modern website” with respect to Adobe Contribute.

Why isn’t Contribute equipped to handle large scale (~10,000+ pages) sites? Contribute doesn’t really have the tools to do anything with regards to content reuse across a site. So as a result, there’s no way to develop interactivity (well, really, you can’t develop anything with it, it’s not a developer tool). You can also forget about getting fancy by integrating things like RSS feeds, or dynamic content in any useful ways (consider, Department X wants a list of their courses for the semester, if they are copy/pasting, there’s no way to control that content once they have plugged it in, which hurts when they totally forget about it the next semester). Contribute is best at static content, on static pages. One page at a time. The newest version (CS3) has done marginally better, in that you can at least paste HTML source code now, but the actual audience that Contribute is aimed at won’t really find that useful. If they knew and understood HTML well, they’d be using Dreamweaver, or at least NVU or something. The crazy part is how good it looks on paper, that idea of simple content management. The reality isn’t that good, especially for developers who must then deal with all the deadwood Contribute leaves behind as things get updated, removed, etc (which is substantial). And don’t forget any template changes you have to make, which would have to be filtered into every file, which is very time consuming (we use SSI (Server Side Include) templates to help stave that issue off, but then that has also caused certain bugs preventing people from creating things like bulleted lists. Craziest thing I’ve ever seen).

I am of the mindset that Contribute has lost its market. It was a good tool five years ago. The game has changed a lot in that time though. A good CMS does everything that Contribute can do with no more of an end-user learning curve, but with the added bonus of being flexible for use with the better power users you serve. Contribute doesn’t have any room to scale up that way. Power users get frustrated in it, and basic users just get lost. The key is that a basic user is a basic user. Period. No matter how simple the software seems on paper, you still have to train them before they can use it, so you might as well give them a tool that not only does things easily, but does them right. For instance, workflow is a joke with Contribute, and as a result page management becomes nearly impossible (and in turn confusing for basic users). There is no review mechanism at all, so content can quickly become outdated and never addressed down the road. We have departments that have copied information from other parts of the site that is out of date by years. This is because they haven’t had the tools to do it correctly in the past.

Like I said, Contribute is designed to do one thing very well, edit static content on static pages. If that is all you want, go nuts, but try anything beyond that and it’s just a bad tool for the job. And in today’s web, a “modern site” is one that generally does not rely on static content this way.  Moreover, a “modern site” is one that also provides current, accurate, fresh data. If you have no ability to keep up on your content in some way, you are setting yourself up for failure. Anyone managing a large site knows that you can’t rely on the editors to simply take it upon themselves to review content (assuming that’s not their primary job).  People rely on web sites now, it’s one of their first stops when they want information on something. If they lose faith in the site as a tool to retrieve accurate information on the subject they want, then you lose a customer. The crazy, lock-me-up-I’m-going-cuckoo goal of a “modern website” is therefore to be omniscient in regards to their audience. You must have a current and correct answer to every question your visitors can and will ask. Totally impossible, I know, but it’s what the audience expects, and there are a lot of ways we can certainly fake it with current web technologies. I don’t feel Contribute is up to that kind of job (not by a long shot).

(Caveat: this is all based on my personal experience in our environment with ~70 Contribute users. We do not run the Contribute Publishing Server.  No doubt others might have more positive opinions.)