SuperSatellite

A Day with dotCMS 1.6.5

// December 2nd, 2008 // 5 Comments » // Software, Tech, Web

It was long coming and well worth waiting for.  On Halloween dotCMS released version 1.6.5, and feeling confident in the code base release, we finally tested and rolled out the upgrade on our production servers (I try not to jump on updates right away, just in case there are any surprise bugs that need to be worked out).  Naturally, we tested things out on our dev box first, were satisfied, and just before Thanksgiving we finalized the upgrade on our live server.  I can say with confidence that this upgrade has proven to be a nice kick in the butt, without a single hitch so far.  Even the improved edit mode interface hasn’t drawn any complaints for being “different.”

A peak at the new dotCMS 1.6.5 edit mode

I’m among the people who think that 1.6.5 is really more like a 1.7 in that it’s not just a minor patch of the 1.6 codebase – there’s a lot of new crap.  Among things to look for:

• WebDav is all new
• Edit Mode is all new
• Widgets have been added
• Updated TinyMCE
• Caching is now in memory instead of disk
• Lucene index has been moved
• PDF page output macro
• A whole crap ton of other stuff

All in all, it’s like 322 different changes, fixes, and additions. Nothing to sneeze at.  And it all matters.  First off, compared to 1.6.0.10, it’s substantially faster on both the front and the back end thanks to the improved caching.  And it flat out blows away 1.5, so if you’re still lingering on that version, I suggest an upgrade pronto.  The speed is coupled with a less noisy log as well, making it easier to tell actual errors from random chatter from the system.  They went through and completely refactored the WebDav interface, so if you use it for bulk adding files or live editing, you should notice a substantial boost in speed there, as well as the ability to autopublish files uploaded through it (great for photos for photo galleries, or live editing things like CSS (Cascading Style Sheets) or VTL files).  The upgrade process itself didn’t feel overly painful either.  One thing that has been a constant complaint was that the system was a little overly painful to actually upgrade, especially from much older revisions.  What I can say is that going from 1.6.0.10 was about the fastest and smoothest upgrade I’ve done so far (that wasn’t just a simple maintenance patch).  SVN update, copy back some server settings, ant clean and deploy, reindex, and we were pretty well off and running (on top of that, I was also switching SVN branches as well).  I could have probably even gotten away with using my old Lucene index, but, fresh system seemed to call for a fresh index.  Interesting side note, I was talking with Chris Falzone over at Edinboro, and he thought upgrading from 1.5 straight to 1.6.5 proved easier than going from 1.5 to 1.6.

My apologies for no real stats on this on my end, but prior to rolling out the upgrade live, I did do some load testing against our 1.6.5 test server using Apache JMeter, and the results were excellent.  I was able to run up about 100 threads within a 30 second time period without drops.  Actually, the machine I was testing with crapped out before the server in some cases.  I also saw some stats that Jason, their lead developer, shared on the mailing list after testing it with Siege that I’ll pass on:

I wanted to share some performance numbers with you.

The test was run on a server with 8 GIGs Ram Dual Quad-Core Xeon E5320 / 1.86 GHz processor running the latest Ubuntu LTS. The server had Postgres 8.3 install locally and Java 5 64bit.

dotCMS configuration
JVM 1536M of memory for the start and max of the heap. The connector was set to 380 Max thread and the Db had 390 max db connections.

The benchmark tool used was siege. We ran siege on 2 separate boxes each with 180 concurrent connections and 0 delay between the connections, so the traffic was a full 360 users attempting to hit the server throughout the entire test. For those who use siege this was the command siege -f urls.txt -r 100 -c 180 -b -i. I will attach the urls used for the test. They were a mix of urls from a starter site. The test was of course performed on our lan. There were 3 switches between the server and the machines running siege.

The total test results were
Total Transactions : 35987
Availability : 99.97%
Average Response time : 0.85 seconds
Total Concurrency : 313.12

Here are the individual numbers from each siege machine.
Transactions: 17996 hits
Availability: 99.98 %
Elapsed time: 97.50 secs
Data transferred: 145.27 MB
Response time: 0.85 secs
Transaction rate: 184.57 trans/sec
Throughput: 1.49 MB/sec
Concurrency: 157.58
Successful transactions: 17996
Failed transactions: 4
Longest transaction: 28.39
Shortest transaction: 0.00

Transactions: 17991 hits
Availability: 99.95 %
Elapsed time: 97.65 secs
Data transferred: 145.22 MB
Response time: 0.84 secs
Transaction rate: 184.24 trans/sec
Throughput: 1.49 MB/sec
Concurrency: 155.54
Successful transactions: 17991
Failed transactions: 9
Longest transaction: 29.46
Shortest transaction: 0.00

We also have some similar numbers with 1.6.5 running on VMWare ESX Server. We will be sharing them soon here. Need to gather all the compiled data for it and specs for the test.

On of the two big, important updates you’ll see a lot of is the much improved edit mode interface, seen above.  Gone are the clunky, vertically columned edit, preview, and live mode buttons.  They’ve moved to the top, and been replaced by a clickable side bar that lets you hide the whole side console completely.  Some much needed UI facelifting was applied with nicer icons and a splash of color.  On the page, container interfaces are squared up nicer, and dropdowns were (mostly) eliminated in favor of one click editing, sorting, and deleting.  That doesn’t sound like much, but if you did much with Flash, you probably fell prey to the dreaded layer monster, where the popups would pop “under” the Flash interface, making it hard or impossible to click the options there.  All in all, the changes were largely cosmetic, somewhat functional, and much needed.  It just feels a little more put together now.

Example of the Widget Modal Window

The other big change is somewhat related, and that’s the addition of widgets.  Widgets have been discussed before, usually as pieces of Web Page Content that are just designed to be dynamic.  This is all well and good, but it didn’t really allow for proper segmentation of functional elements of a site.  Now, widgets are properly defined with a structure.  You set them up like structured content, except your fields basically serve as the variables for a macro.  Think of it as a UI for macros.  The primary difference being a macro can be embedded in content, while a widget goes in place of content.  Why this is so awesome is because one, like I said, it allows you to properly segment dynamic tools for your site, and two, even though a macro is only like a line of code, we all know that for some people one line is just one line too many.  This let’s them fill out a simple, non-threatening form, and see results.  The interface itself is cool, as adding a widget brings up a nice little modal window with all the widgets defined available right there in the page.  Plus, any container can be widget enabled, so that you can make containers fairly multipurpose, in the event that you want one container on a particular page to do something different, but still adhere to a basic layout option.

Speaking of the modal window, I would love to see this applied throughout the site to get rid of popups all together.  Get rid of the content search popup forms for reusing content. Use a modal window for quick adding or editing of content.  It makes people feel connected when they don’t physically have to juggle windows, especially when in some cases (like adding a picture to content), the popup adds information to a different popup below it, and it doesn’t go away when you click it, so you don’t actually get any feedback from the system that a change has taken place.  So there’s a wishlist item from me.

What makes this so worthwhile is that there is a definite maturing of the code base taking place.  The attention to the UI of edit mode, while not changing substantially in a functional way so much, shows dotCMS becoming a much more professional system that cares as much about the user experience as the functionality.  This is an excellent sign as we get ever so closer to what should be a real dam breaker of a version 2.0 next year (hopefully), and the rate of progression is fairly steady.  Like I said, rather than being 1.6.5, this would really almost qualify as a 1.7 in my books, but hey, I don’t do the versioning.

Extending the pageContent() Macro in dotCMS

// September 19th, 2008 // No Comments » // Software, Tech, Web

Here’s my latest nugget of info to help you extend dotCMS.  This isn’t anything complex by any stretch of the mind, but it should be a little time saver for those of you looking into doing blogs, archives, or anything else where you have several pages of content to flip through.  By default, dotCMS comes with a nice little macro called pageContent() that allows you to pull in a list of content, and get a set grouping of results based on where they fall in the sort order.  The only failing was that you had to write your own way to go back and forth with results.

No longer!  With this simple, handy macro, you can automatically get forward and backward navigation after you’ve written your foreach() loop for the pageContent() results.  It makes use of a few of the variables that are created when you run pageContent().  You can supply your own text for the navigation links, as well as give it a custom class name that will automatically be applied to all the elements so that you can format it as you wish.  The code for the macro itself follows below the example.  Here’s a sample of it in use, and what it looks like:
#set($showPageNum = "true")
#set($pageLinkVar = "year=${yearVar}&")
#pageContentNavigation("$page")

Example of the navigation macro

Here’s the code to get you going.  Just paste it into your /[$DOTCMSROOT]/dotCMS/WEB-INF/velocity/dotcms_library_ext.vm file, or wherever else you’re loading your custom macros, restart the server (if needed), and get going.  There are any number of ways you can modify it to suit your needs, so go nuts!  You can copy the code below, or download the pageContentNavigation_macro.vm file.

Gettin’ Your Macro On

// July 29th, 2008 // 4 Comments » // Scripts, Software, Web

I think it’s time to rock out with another dotCMS blog, and dig into some basic stuff that can be very useful to people getting started in the content management system.  There was a topic recently on the mailing list that was asking about where to look to see some examples of macros, and where you can save your own.  This looked to be a good topic to cash in on (speaking of cash, wanna loan me some?).

Macro output on a page

In dotCMS, a macro is basically like it is in other applications: it’s a collection of code one calls to reproduce complex processes in a simple, and straightforward fashion.  You know, to help stupid people out.  To put it in other terms, it’s effectively what one might call a function in other languages.  You give it a name, set parameters it can take, and define some output.  dotCMS comes out of the box with a lot of macros you can readily use, sometimes without even realizing it.  These include things like the commonly used pullContent(), and fancier tools like videoPlayer().  Naturally, dotCMS has a list of these built in macros, and how to use them.  If you want to take a look at example code, the built in macros are stored in /[dotcmsroot]/dotCMS/WEB-INF/velocity/dotCMS_library.vm.  Don’t mess with those though, because odds are you’ll break stuff.  Just trust me on this.

There are a couple ways to approach writing your own macros.  The first is to write a second .vm file, and store it with the default one in the velocity folder.  It will be loaded when the server starts, but requires a server restart when you change or update them, because they are cached in the system when it starts.  Conventionally, the file name dotCMS_library_ext.vm is recommended.  By convention, I just mean that’s what the dotMarketing programmer folks have recommended to me.  The other way is to use a static container, which you include first thing in all the templates you want the macros to be available to.  Macros edited and saved this way do not require server restarts to use, but naturally you must make sure you’ve included the container in EVERY template where you suspect your macros might be used (and also ones where they might not. Trust me, never underestimate the stupidity of your users).  Neither method is right or wrong, and you should just choose the way that fits your development style best.  I’m still looking for a solution that would qualify as chaotic neutral, however.

Okay, so naturally you need to have something of a working knowledge of Velocity code to write a macro, so if you’ve never taken the time to learn some, I suggest you go do that.  Mr. Falzone has talked about this issue, so go get yourself some learnin’ and stop trying to run before you can walk.  Now, get off my lawn. If you already know Velocity stuff, go anyway, he’s a nice guy and worked hard, so you owe it to him to read it and pretend like you’re new to the subject.  Don’t worry, it’ll be our little secret.  But, assuming you understand the basics, this should be pretty straightforward.  So, starting up, we need to define the macro.  I’ll use as an example the macro I wrote that pulls in a list of events from the calendar by a tag:

So, we start by calling the macro() function, naming it, and giving it a list of variables to expect.  You can have more than one variable, just list them one after the other (no commas or anything).  As you can see, before I open the macro, I list a couple variables in comments that aren’t required, but can be set before you use pullEventListByTag() and they’ll be pulled in later.  Next, I’m going to set three variables that I’ll use when querying for events, then I’ll build out the query and set it as a variable, and set how I want it ordered (date1 is the event start date).

Okay, next up I’m going to check for those optional variables, and if they aren’t set, it will give some default values.  After that, I’ll call the pullContent() macro (as you can see, you can call other macros within macros), and save the results in its own list variable.

That takes care of the heavy lifting.  Next I’ll start the output.  In this case I’ll build out a <div> to hold the contents, pump out a header, and start a standard foreach() loop for the results in $eventMacroList just as we would when using the pullContent() macro like any other time.  I told you this wasn’t hard.

And believe it or not, that’s it. Once this is saved and initiated in the manner you choose, you can call your custom macro with the parameters you’ve set, and it will execute the code. In this case, you could call pullEventListByTag(“football”) to return all events tagged football. On another page you could call pullEventListByTag(“baseball”) and get a different set, but still get the same structured output.  Such is the nature of macros.  Any process you would need to do over and over again, or need to simplify for less savvy users are prime candidates for macroification.  In simple terms, this example is nothing more than a macro to automate a pullContent() call and foreach() loop.  If you look at the default macro file in dotCMS, you’ll see how much more complex you can get with them.

So, the complete macro for this example would be as follows.  Feel free to copy and modify it to fit your own needs, or to play around with to get used to the process.  I’d be happy to see how you improve or modify it.  You can also discuss it in the forums, or share your own creations.

Overview of the New dotCMS 1.6 Calendar

// May 14th, 2008 // 7 Comments » // Software, Web

With the release of dotCMS 1.6 came one of the more anticipated new features in recent months, their new “social” calendar system. This calendar represented a noted improvement over the previous event portlet (though the event portlet still exists for some additional functions, like scheduling and managing conferences, etc). This new calendar makes use of a portlet to manage the system in the backend similar to the old event system, and builds out on the front end through a combination of Velocity, .vtl files, and some layout bits. It also creates event entries as content in a structure, so that it can be queried and pulled no different from any other content in the system (note that you might want to make sure that you don’t currently have structures called “Event,” “Building,” or “Facility” already, as it will want to make them if you are upgrading. New installs obviously don’t need to worry about that).

If you want to take a look at the calendar first hand, you can see the example that is up at the demo site, at http://demo.dotcms.org/calendar/. To access the back end, login to the admin console by going to http://demo.dotcms.org/c and plug in the user ID test@dotcms.org and the password test. In there, you’ll see the tab for “Calendar” that you can click on that will take you to a similar, but stripped down looking version of the calendar. I’ll be using the demo’s site front and back end when describing features, and in the video below.

Part of the major improvement with this system is that as opposed to the event portlet, it provides true calendar functionality, whereas the event system provided more of a list view, regardless of how you tackled it. This calendar is very AJAX (Asynchronous Javascript And XML) heavy, and thus can present an accessibility problem by default, but since all calendar events are simply structured content, you can very easily create a standard HTML based variant for users with special screen needs (and really, do you know a calendar that is truly 508 compliant out of the gates in the first place?). Storing events as structured content gives you an extra layer of control over the presentation of those events, control that is unmatched in other calendar systems. The interface itself is fairly smooth and intuitive, without a lot of tricky links, or confusing navigation (I’m look at you, Active Data Calendar). Though fairly simple in terms of usage, the fact that everything just works, and works like it should, carries a high value. And in keeping it simple, it is actually a very powerful little tool.

The calendar itself supports all the traditional types of functionality: Time, date, place, multi-day, all-day, list-week-month views, RSS feeds, detail view and searching. Additionally, you can add events from the front end (if enabled), tag events, filter by tag(s), filter by calendar(s), attach files and images, provide detail popups, and provide automatic integration into Google Maps. Using categories, you can effectively create completely self contained calendars of different events, and one event can exist within multiple calendars simultaneously. All in all, it’s a pretty nice feature set. It will also save you trying to integrate with a third party calendar for your business that would require and supply it’s own login, back end, and functionality challenges. With some CMS (Content Management System)’s, they might have a calendar, but it’s an additional install, plugin, or module that you have to manage. dotCMS includes the calendar out of the box (still free), and if you don’t want to use it, you just don’t display the page in the back end, so there’s no extra labor involved. Other enterprise CMS’s come with built in calendars, but you’ll pay dearly for those packages.

The primary challenge right now is that if you do not load the starter site, or are upgrading from 1.5.x, you don’t have all the front end files (not to be confused with the back end admin portlet for the calendar, which is totally there whether you upgraded or installed fresh and will be in need of no special attention) that you’ll need to display the calendar (this will be changing soon, however, as I understand they plan on bundling all that with the CMS so you’ll have it if you upgrade or don’t use the starter site). What I did to solve this problem was to go to the demo site, and lift all the files out of their /calendar directory. This had one disadvantage: all the files are hardcoded to the /calendar directory. So, what I have done is taken all these files, and modified them so that you can upload them into your system, and they will adapt to wherever you want to put it. Just download this zip file which contains everything needed to display the calendar to your visitors, extract it, and upload it either with the multi file uploader or over WebDav. The only other thing you need to do then is make a page in whatever folder that will be your calendar, let’s just call it index.dot, and place the following code into a piece of web page content that you use on the page:

This will load and fire off all the rest of the Velocity templates. There are two CSS (Cascading Style Sheets) files (grids-min.css and reset-min.css) that are needed because the calendar was designed with Yahoo Grids, and those styles make sure everything lines up properly. A third CSS file (cal-base.css) that is included is the base starter site CSS, and you could remove it as necessary to meet your styling needs, but I included it at the moment for the sake of appearance reproduction. All three CSS files are included in the zip file above already, and you’ll see them referenced in velocity/calendar.vtl. Using the files from the zip file, and the content above, you now have a calendar that you can move to a new directory should the need arise, or should you simply not want it in /calendar. If you move it, just make sure you change the /absolute/path/to in your piece of content to point at the new location of the load-calendar.vtl file. You could even make that portion dynamic if you wanted to, simply by removing the first line from load-calendar.vtl, and adding it to the content, making it:

The only reason I didn’t do that by default was so that way you never have to copy and paste that set() from here (since I can’t “include” a piece of content in the zip file), and to keep the information going into the content item that tells it to make the calendar as simple as possible. Odds are, once you place the calendar, you won’t be moving it around anyway. But this way you don’t have to change all the paths yourself, and you aren’t locked into using /calendar.

Below, I have included a brief overview of the new calendar, so that you can see some of the functionality and how it all comes together.

Download the customized dotCMS social calendar front end files (.zip, 131KB)

The Prodigal Son Returns

// May 8th, 2008 // No Comments » // Software, Travel

Finally back and settled in after returning from training in Miami. Now just very busy trying to catch up, and put some of the things that I learned to use. I did manage to grab some photos around the area on CocoWalk where I was, so feel free to browse the Flickr set. I also started playing around with geotagging the photos. I have no idea how useful that will be in the future, but what the heck? I wish that I’d had more time to get out and about, but I did hit two excellent sushi places, instantly reminding me that what I pass off as sushi here is nothing but a sick and twisted joke by comparison.

First, how come no one told me that some Chinese company was making new MGs? I saw one at the condo where I was put up at. It wasn’t anything fancy as far as sport convertibles go, but it was still pretty neat in its own way. Now I want to drive one and see what they feel like.

Nearby was CocoWalk, which is where the dotMarketing headquarters are and where I spent a lot of my time. It was Cinco de Mayo the night I arrived, so it was a pretty popular site, with the shops and restaurants (though I made the mistake of passing up Hooters.  I’m not sure what I was thinking to make that mistake). However, I did take a couple hours the night I got there and saw Iron Man. Let me just say, I was absolutely not disappointed. Go see it.

On the travel side, I think I will make the effort to never fly US Airways again. Worst planes and service I’ve ever dealt with. Tiny pretzels, mean spirited, inattentive stewardesses, terribly seats, and small planes. And to top it off, I appear to be getting increasingly worse anxiety while driving long distances by myself. My nerves light up high enough on the drive to KC and back that it very nearly makes me sick. I’m not sure what to do about this yet, but it is very unsettling. Obviously drugs would probably impair my ability to drive, and if I had someone drive me, it would cost a fortune. I think I’m going to try to start flying out of Tulsa, which is a tiny bit closer, or even Joplin, assuming I can find well timed hoppers.

As for dotCMS, I feel much better equipped than I have in the past after opening my skull and letting their lead developer pour knowledge into it for a full day. I anticipate having no problems setting up my development environment soon, and upgrades from here on out should be much, much easier. I also understand better how the parts come together, and what they do when the break. I also agreed to devote some time to helping with their community site doing some writing or theme crafting. That will be a fun side project I think.

I know some people have had questions about setting up dotCMS to run in an Eclipse development environment. If you were like me, you might have been having problems because you were reading the wrong setup documentation. Turns out there is a new one that was misfiled that is much better and up to date. Also, if you are running a dotCMS site on MSSQL, don’t run ant buildsql when you update. It breaks things, because it’s meant for the Oracle and Postgres camps. You’ll want to use ant buildmXsql.  Learn from my mistakes, heh.