Siim Viikman's blog

In my project I have many small JavaScript files containing mostly jQuery plugins. Also there are many CSS files, because I prefer to split my CSS declarations to multiple files based on some context. All this results in many static files that must be loaded in every page load which increases latency.

So I decided to combine multiple files into one single file and also minify it. I looked for some libraries that would get the job done for me and I stick with YUI Compressor which requires Java runtime. I still want to step through readable javascript when I’m developing the app so I decided to combine and minify files only when I’m compiling in release mode, but we will get to that point later on. There were two action I needed to take to achieve this. I needed to modify the web project to include after build action for combining files and to write some kind of extension that’s either loads debuggable source files or compressed files. None of the steps were hard.

Post build step is actually two actions – temporarily combining multiple files into a single file and then minifying that file with YUI Compressor. It gets triggered only when building in a release mode. All the required tools including YUI Compressor are included in VCS so that all dependencies are there when developer checks out code.

<PropertyGroup>
	<yuiCompressor>java -jar ..\..\Tools\yuicompressor\yuicompressor-2.4.2.jar</yuiCompressor>
 </PropertyGroup>
 <Target Name="AfterBuild">
	...
	<CallTarget Targets="Minimize" Condition="'$(Configuration)' == 'Release'" />
 </Target>
 <Target Name="Minimize">
	<GetAssemblyIdentity AssemblyFiles="bin\$(ProjectName).dll">
	  <Output TaskParameter="Assemblies" ItemName="assemblyInfo" />
	</GetAssemblyIdentity>
	<ItemGroup>
	  <JqJsFiles Include="Content\Scripts\jquery.*.js;Content\Scripts\json2.js;" />
	</ItemGroup>
	<ItemGroup>
      <OldVersions Include="Content\Scripts\jq.plugins.min-*.js" />
    </ItemGroup>
	<!-- js Merge and Minimize -->
	<ReadLinesFromFile File="%(JqJsFiles.Identity)">
	  <Output TaskParameter="Lines" ItemName="jqJsLines" />
	</ReadLinesFromFile>
	<WriteLinesToFile File="Content\Scripts\jq.merged.js" Lines="@(jqJsLines)" Overwrite="true" />
	<Delete Files="@(OldVersions)" />
	<Exec Command="$(yuiCompressor) --type js &quot;$(ProjectDir)Content\Scripts\jq.merged.js&quot; -o Content\Scripts\jq.plugins.min-%(assemblyInfo.Version).js --charset utf-8" />
	<Delete Files="Content\Scripts\jq.merged.js" />
</Target>
<PropertyGroup>
	<CopyAllFilesToSingleFolderForPackageDependsOn>
		CollectMinifiedFilesInPackage;
		$(CopyAllFilesToSingleFolderForPackageDependsOn);
	</CopyAllFilesToSingleFolderForPackageDependsOn>
</PropertyGroup>
<Target Name="CollectMinifiedFilesInPackage" Condition="'$(Configuration)' == 'Release'">
	<ItemGroup>
	  <_JsFiles Include="Content\Scripts\jq.plugins.min-*.js" />
	  <FilesForPackagingFromProject Include="%(_JsFiles.Identity)">
		<DestinationRelativePath>Content\Scripts\%(Filename)%(Extension)</DestinationRelativePath>
	  </FilesForPackagingFromProject>
	</ItemGroup>
</Target>

As you can see, I include assembly version number (which is generated by build) in the names of the compressed files. This way I can always refer to correct JS files which are accordance with the given assembly. There’s also a target called CopyAllFilesToSingleFolderForPackageDependsOn which is required to include those compressed files in the MSDeploy package because they are not included in the web project and therefore not automatically included.

For the other part I wrote a partial view to load CSS & scripts files based on the configuration. And I use the partial view in my layout files (or wherever I need) to include all required resources. A part of the view:

@if (HttpContext.Current.IsDebuggingEnabled)
{
	@Html.Script("~/Content/Scripts/jquery-1.4.4.js")
	@Html.Script("~/Content/Scripts/jquery-ui-1.8.9.custom.min.js")
	@Html.Script("~/Content/Scripts/jquery.ui.datepicker-en-GB.js?" + DateTime.Now.Ticks)
	@Html.Script("~/Content/Scripts/jquery.selectboxes.js?" + DateTime.Now.Ticks)
	@Html.Script("~/Content/Scripts/jquery.tooltip.js?" + DateTime.Now.Ticks)
	@Html.Script("~/Content/Scripts/json2.js?" + DateTime.Now.Ticks)
}
else
{
	@Html.Script("~/Content/Scripts/jquery-1.4.4.min.js")
	@Html.Script("~/Content/Scripts/jquery-ui-1.8.9.custom.min.js")
	@Html.Script(string.Format("~/Content/Scripts/jq.plugins.min-{0}.js", Html.AssemblyVersion()))
}

Side note: I append the timestamp at the end of the file names to prevent any caching issues in development environment

I make the decision which files to load based on the debug configuration in the web.config, not based on the build configuration in the compile time. I also deploy all the resource files (compressed and original) so I am able to switch between them easily, without recompiling and deploying.

That’s it. No custom out-of-the-solution build scripts needed. And the solution can be also applied to ASP.NET Webforms applications with little modification in including files.

When I am developing MVC applications I see that there are some common behavior for certain types of actions (eg. list view, edit view) which means similar code. A common solution to that problem is to create a base controller with that behavior. And that’s also how I tried to solve the problem first. I created base controllers for each view type and their related actions. But it created new problems.

I ended up with a multi-level inheritance tree which made it difficult to track down things. It wasn’t possible to look at the controller and simply say what it is doing. Some of the actions were defined at the upper level that required implementations at the lower level and so on, which created a lot of jumping between classes.
And then there was a common inheritance problem – I needed some behavior from base controller but not all, so I had to implement methods I really didn’t need. Or I needed behavior from multiple base controllers.
Third, it created a big constructors in controllers with all different dependencies required by base controller classes.

I thought about using approach called controllerless actions, but after a little research I found it is too complicated with a given structure. Luckily I can create custom action results that can encapsulate all the logic and data for returning a result for given action. But then I found this post which described separating WHAT of an action result from the HOW. By that I mean that my custom action result contains all the required information to execute an action, but the execution logic is in different object – in an action result invoker I created.

By using this approach I was able to separate all that execution logic from base controllers to separate action invoker. And controller was only responsible for returning a proper action method result with all relevant information based on the input. No repetitive execution logic anymore. For that, I created a base ActionMethodResult which is so called marker base class for my custom action method results.

public abstract class ActionMethodResult : ActionResult
{
	public override void ExecuteResult(ControllerContext context)
	{
		throw new InvalidOperationException("Action should be executed through action invoker");
	}
}

A one simple action method result implementation looks like this. It contains a task to execute and a result to return when task execution was successful and other one for failure cases.

public class TaskActionResult<TEntity> : ActionMethodResult
	where TEntity : IEntity<int>
{
	private readonly Func<ITask<TEntity>> _task;
	private readonly Func<TaskResult<TEntity>, ActionResult> _successContinuation;
	private readonly Func<TaskResult<TEntity>, ActionResult> _failureContinuation;

	public TaskActionResult(Func<ITask<TEntity>> task, Func<TaskResult<TEntity>, ActionResult> successContinuation, Func<TaskResult<TEntity>, ActionResult> failureContinuation)
	{
		_task = task;
		_successContinuation = successContinuation;
		_failureContinuation = failureContinuation;
	}

	public Func<ITask<TEntity>> Task
	{
		get { return _task; }
	}

	public Func<TaskResult<TEntity>, ActionResult> SuccessContinuation
	{
		get { return _successContinuation; }
	}

	public Func<TaskResult<TEntity>, ActionResult> FailureContinuation
	{
		get { return _failureContinuation; }
	}
}

But it’s not the case with all action results. Some of them just take some inputs and invoker decides how to respond to that. The main point is the same – invoker generates some response based on the given action method result in the context of current request. Action invoker for TaskActionResult is like this.

public class TaskActionResultInvoker<TEntity> : ITaskActionMethodResultInvoker<TEntity>
	where TEntity : IEntity<int>
{
	public ActionResult Invoke(TaskActionResult<TEntity> actionResult, ControllerContext controllerContext)
	{
		if (!controllerContext.Controller.ViewData.ModelState.IsValid)
		{
			return actionResult.FailureContinuation(null);
		}

		var result = actionResult.Task().Run();
		if (!result.ValidationResults.IsValid)
		{
			result.ValidationResults.AddToModelState(controllerContext.Controller.ViewData.ModelState);
			return actionResult.FailureContinuation(result);
		}

		return actionResult.SuccessContinuation(result);
	}
}

Then I created a custom IActionInvoker where I overrode the CreateActionResult method to create a proper ActionResult based on my custom action method result. Each of my action invoker held it’s own dependencies so I needed to create a façade for that which got my action invoker as a constructor argument so it could be resolved by my container. The creation of the facades and action invokers is a bit tedious, but it was a compromise I was willing ta make.

public class InjectingActionInvoker : ControllerActionInvoker, IActionMethodResultCreator
{
	private readonly IServiceLocator _serviceLocator;

	public InjectingActionInvoker(IServiceLocator serviceLocator)
	{
		_serviceLocator = serviceLocator;
	}

	protected override ActionResult CreateActionResult(ControllerContext controllerContext, ActionDescriptor actionDescriptor, object actionReturnValue)
	{
		if (IsActionMethodResult(actionReturnValue))
		{
			var result = CreateActionMethodResult(actionReturnValue, controllerContext);

			return IsActionMethodResult(result)
				? CreateActionResult(controllerContext, actionDescriptor, result)
				: result;
		}
		return base.CreateActionResult(controllerContext, actionDescriptor, actionReturnValue);
	}

	public ActionResult CreateActionMethodResult(object actionReturnValue, ControllerContext controllerContext)
	{
		var wrappedResultType = CreateActionInvokerFacadeType(actionReturnValue);

		var invokerFacade = (IActionMethodResultInvoker)_serviceLocator.Resolve(wrappedResultType);

		return invokerFacade.Invoke(actionReturnValue, controllerContext);
	}

	private static Type CreateActionInvokerFacadeType(object actionReturnValue)
	{
		var actionMethodResultType = actionReturnValue.GetType();

		Type openWrappedType;
		Type wrappedResultType;

		if (actionMethodResultType.IsGenericImplementation(typeof(TaskActionResult<>)))
		{
			openWrappedType = typeof(TaskActionResultInvokerInvokerFacade<>);
			wrappedResultType = openWrappedType.MakeGenericType(actionMethodResultType.GetGenericArguments()[0]);
		}
		// code for other types of action results removed for brevity
		else
		{
			openWrappedType = typeof(ActionMethodResultInvokerFacade<>);
			wrappedResultType = openWrappedType.MakeGenericType(actionMethodResultType);
		}

		return wrappedResultType;
	}
}

Note the recursive call to the CreateActionResult method. It is because my action method results my return other action method results for some execution paths, which in turn need to be invoked with their own type of invokers. So invokers are executed so long as result is “regular” ActionResult which can be executed using ASP.NET MVC built-in behavior. 

Finally, when all hooked up, my controller action looks like this. Simple, clean and without inheritance.

[HttpPost]
public ActionResult Edit(int id, ViewModel item)
{
	item.Id = id;

	return new TaskActionResult<Entity>(
		() => _taskFactory.ForSave(() => CreateEntityFromViewModel(item)),
		success => SaveSuccessResult(success, "Task executed!"),
		SaveFailureResult);
}

Now I’m able to create much more simple controllers in the maintenance point of view, they are better to follow. It also improved testability. Because in my opinion, inheritance is pain in the butt regarding testing. This approach allowed me to easily test the result of the controller action (which is basically a plain data) separately from the execution of that result. Which I can do in an isolation compared to the approach when I was using controller inheritance which also resulted in some sort of tests inheritance.

Now and then there comes a need to save some documents to the database also, along with the other information stored there already. It’s not a big deal to save some additional files to the database, especially when it’s not the main purpose of the application, just some extra. So no need for different storage mechanism.

This time I’m using NHibernate and I thought how I should structure tables and write mappings so that files data gets loaded only then when I really need it (eg. downloading a file). I also save some metadata about the file, some if it has business meaning, some of it doesn’t.

Database

I came up with the following simple schema:

I can load all the related meta-info about the file easily without touching the real content. That kind of schema also allows us to put FILE_DATA table to a different file group in MS SQL Server, so physically it can be located on other drive or partition whereas files metadata is located with our other tables.

Mappings

But this is only database side, there are also NHibernate mappings we have to work on. From NHibernate point of view, it’s not one-to-one relation but more like an one-to-many relation, meaning that File may have 0 or more FileData objects. So internally I mapped it like so, but to the outside I exposed it like one-to-one. I am using FluentNhibernate and here are my mappings for that:

public class FileMap : ClassMap<File>
{
	public FileMap()
	{
		DiscriminateSubClassesOnColumn("FILE_TYPE");

		Id(x => x.Id);
		Map(x => x.Type, "FILE_TYPE")
			.ReadOnly();
		Map(x => x.Name);
		Map(x => x.ContentType);

		HasMany<FileData>(Reveal.Member<File>("_files"))
			.Access.Field()
			.Cascade.AllDeleteOrphan()
			.Inverse();
	}
}

public class FileDataMap : ClassMap<FileData>
{
	public FileDataMap()
	{
		Id(x => x.Id);
		Map(x => x.Content);
		References(x => x.File);
	}
}

And related objects:

public abstract class File : IntegerIdentityEntity
{
	private IList<FileData> _files;

	protected File()
	{
		_files = new List<FileData>();
	}

	protected virtual IList<FileData> Files
	{
		get { return _files; }
	}

	protected virtual FileData FileData
	{
		get { return Files.SingleOrDefault(); }
	}

	public virtual string Type { get; protected set; }

	public virtual string Name { get; set; }

	public virtual string Number { get; set; }

	public virtual string ContentType { get; set; }

	public virtual byte[] Data
	{
		get
		{
			return FileData != null ? FileData.Content : null;
		}
		set
		{
			if (FileData != null && value == null)
			{
				FileData.File = null;
				Files.Remove(FileData);
			}
			else if(FileData != null)
			{
				FileData.Content = value;
			}
			else if(value != null)
			{
				var dataFile = new FileData {File = this, Content = value};
				Files.Add(dataFile);
			}
		}
	}
}

public class FileData : IntegerIdentityEntity,
{
	public virtual File File { get; set; }

	public virtual byte[] Content { get; set; }
}

As you can see, externally there is only one object – File, with property Data which returns the contents of the file as a byte array. Internally it is stored in the FileData collection because it’s the natural way how NHibernate handles such scenarios but that doesn’t mean we must always expose it as collection.

Conclusion

This approach allows to store files and their metadata in a database in a simple way, without adding any extra overload to fetching when it’s not necessary. From a database perspective it don’t add any extra constraints either, because we can put data table to a different partition so keeping the size of the main database reasonable.

In ASP.NET MVC, every controller method which is public and not marked with NonAction attribute is considered as an action. And action name is the same as method name. But you can also provide ActionName attribute with specified action name. There is also different attributes to constrain actions to handle only specified HTTP methods (GET, POST, PUT etc). All that information is used when finding which controller method to execute.

So, when technically (in a sense of C#) you can create many method overloads for a controller method, you end up with runtime exception when you don’t further restrain those actions. For example, consider following controller:

public class ProductController : Controller
{
	public ActionResult Edit(int id)
	{
		return View("Edit");
	}

	public ActionResult Edit(int id, ProductItem item)
	{
		return View("EditPost");
	}
}

public class ProductItem
{
	public int Id { get; set; }

	public string Name { get; set; }
}

When executing Edit action, MVC don’t know which method overload to execute, because current route could be handled by both of the Edit methods. But when decorating first method with HttpGet attribute and the second with HttpPost attribute, it works well.

Now, the trouble comes when you execute one of such methods from the view with RenderAtion method. Lets say you have two controllers:

public class ProductController : Controller
{
	public ActionResult Edit(int id)
	{
		return View("Edit");
	}

	[HttpPost]
	public ActionResult Edit(int id, ProductItem item)
	{
		return View("EditPost");
	}
}

public class ProductDetailController : Controller
{
	public ActionResult Edit(int id)
	{
		return View("EditDetail");
	}

	[HttpPost]
	public ActionResult Edit(int id, ProductDetailItem item)
	{
		return View("EditDetailPost");
	}
}

public class ProductDetailItem
{
	public string Type { get; set; }
}

public class ProductItem
{
	public int Id { get; set; }

	public string Name { get; set; }
}

ProductController is used to display product form view. On that form, I also call RenderAction to render Edit method on the ProductDetailController which displays some product sub-detail form. When executing first ProductController.Edit method, all works fine. When I submit product form, then the second Edit method is executed (because it handles POST requests).

At first, it looked like some weird behavior. But after when I gave a thought how MVC resolves which action method to execute, it all works out. It’s because RenderAction uses the same algorithm to resolve actions as a regular action link. So pay attention when using same pattern with controllers and actions.

Lately I’ve been doing development on ASP.NET MVC and using MvcContrib grid to render simple grids. I have to say, it is very easy to use and extendible in every way. If I feel that something is missing or I want to change some behavior, then I just need to implement the proper interfaces and inject my own implementation for what I need.

For views I use view models with data annotations which specify how some data should be displayed on the view. I wanted to use data annotations also for specifying which properties should be rendered as sortable columns and which is the default one. Grid doesn’t do any sorting by itself, it solely relies on the input (which is a good thing, IMHO).

Fortunately, it was very easy to add. I needed only two things – customize the MVC model metadata provider to include my custom values and create a custom ColumnBuilder for the grid that could read values I included in the metadata. Brad Wilson has a nice post about how to extend model metadata providers. The code for the OrderByAttribute and with the bit that extends model metadata, is here:

[AttributeUsage(AttributeTargets.Property, AllowMultiple = false)]
public class OrderByAttribute : Attribute
{
	public OrderByAttribute()
		: this(SortDirection.Ascending, false)
	{
	}

	public OrderByAttribute(SortDirection sortOrder, bool isDefault)
	{
		SortOrder = sortOrder;
		IsDefault = isDefault;
	}

	public SortDirection SortOrder { get; set; }

	public bool IsDefault { get; set; }
}

public class GridMetaDataProvider : DataAnnotationsModelMetadataProvider
{
	public const string SortableValueName = "Sortable";

	protected override ModelMetadata CreateMetadata(IEnumerable<Attribute> attributes, Type containerType,
		Func<object> modelAccessor, Type modelType, string propertyName)
	{
		var metadata = base.CreateMetadata(attributes, containerType, modelAccessor, modelType, propertyName);

		var orderByAttribute = attributes.OfType<OrderByAttribute>().FirstOrDefault();
		if (orderByAttribute!=null)
		{
			metadata.AdditionalValues.Add(SortableValueName, orderByAttribute);
		}

		return metadata;
	}
}

Next I need to create a column builder, similarly to AutoColumnBuilder which comes with MvcContrib. Unfortunately, I cannot extend that class because there are no extension point, but I used it as a guidance. For each property from metadata, I execute the following code, to apply sorting metadata:

var isSortable = property.AdditionalValues.ContainsKey(GridMetaDataProvider.SortableValueName);
column.Sortable(isSortable);
column.SortColumnName(property.PropertyName);

That’s all about building columns from metadata. Next thing I need to do, is to tell the grid which column is used for initial sorting. Because grid wants object of type GridSortOptions as an input, it is created by the controller. So I wrote a little extension method to populate GridSortOptions from model metadata.

public static GridSortOptions ApplyDefault<TItem>(this GridSortOptions sortOptions) where TItem : ListItemBase
{
	// When sort options is specified, don't apply default values
	if (sortOptions != null && !string.IsNullOrEmpty(sortOptions.Column)) return sortOptions;

	var property = typeof (TItem).GetProperties().Where(ContainsDefaultOrderBy).FirstOrDefault();
	if (property == null) return sortOptions;

	var newSortOptions = sortOptions ?? new GridSortOptions();
	newSortOptions.Column = property.Name;
	return newSortOptions;
}

private static bool ContainsDefaultOrderBy(PropertyInfo property)
{
	var orderBy = (OrderByAttribute)property.GetCustomAttributes(typeof (OrderByAttribute), false).FirstOrDefault();
	return orderBy != null && orderBy.IsDefault;
}

That’s it. Now you can decorate your view model with attributes and grid takes care of the rest.

Joomla has a debug plugin that is able to display different kind of information about page execution – load times in page lifecycle, strings localizations and executed SQL statements. But when using Doctrine, you can’t use this Joomla plugin for SQL statements because Doctrine uses it’s own database connection management, different from Joomla’s.

Fortunately Doctrine already comes with built-in profiler support, we only need to integrate it with Joomla plugins framework. Joomla documentation contains a tutorial on how to create a plugin, it contains also a list of events that are supported by the plugin framework. And there are a wiki page that describes the Joomla API execution order.

I needed onAfterDispatch and onAfterRender events so I created plugin of type system. In onAfterDispatch event I attach Doctrine profiler to the database connection and in the onAfterRender event I collect the SQL statements. Profiler logs different kind of events, but I’m only interested in events that show execution.

Here is the source of the plugin ( I have skipped the not so important parts to keep it shorter).

class plgSystemDoctrine_profiler extends JPlugin
{
	private static $NewLineKeywords = '/(FROM|LEFT|INNER|OUTER|WHERE|SET|VALUES|ORDER|GROUP|HAVING|LIMIT|ON|AND)/';

	private $_outputEnabled = false;
	private $_log4phpEnabled = false;
	private $_logEnabled = false;
	private $_logName = 'doctrine.profiler';

	private $_profiler = null;
	private $_formattedEvents = array();

	public function __construct(&$subject, $config)
	{
		parent::__construct($subject, $config);

		// Initialize from params
	}

	private function getProfiler()
	{
		return $this->_profiler;
	}

	public function onAfterDispatch()
	{
		$this->_log4phpEnabled = $this->_log4phpEnabled && class_exists('Logger');

		if (!class_exists('Doctrine_Connection_Profiler')) return;

		$this->_profiler = new Doctrine_Connection_Profiler();

		$connection = Doctrine_Manager::connection();
		$connection->setListener($this->_profiler);
	}

	public function onAfterRender()
	{
		if ($this->getProfiler() == null) return;

		foreach($this->getProfiler() as $event)
		{
			if ($event->getName() != 'execute') continue;

			$this->_formattedEvents[] = $this->formatEvent($event);
		}

		$this->writeOutput();
		$this->writeLog4php();
		$this->writeLog();
	}

	private function writeOutput()
	{
		if (!$this->_outputEnabled) return;

		// write output to page
	}

	private function writeLog4php()
	{
		if (!$this->_log4phpEnabled) return;

		// Write output to log4php
	}

	private function writeLog()
	{
		if (!$this->_logEnabled) return;

		// Write output to Joomla logs
	}

	private function formatEvent(Doctrine_Event $event)
	{
		$sql = $event->getQuery();
		$sql = preg_replace(self::$NewLineKeywords, "\n\t\\0", $sql);

		$formatted = str_repeat('--', 10)."\n";
		$formatted .= $sql."\n";
		$formatted .= "Params: ".join(', ', $event->getParams())."\n";
		$formatted .= 'Elapsed time: '.$event->getElapsedSecs().' seconds'."\n";
		return $formatted;
	}
}

The usage of the onAfterDispatch event depends on your architecture, in some cases it may be too late for you to attach the profiler in that point. Another options is to use similar approach like Joomla is using with its profiler – usage of global variable $_PROFILER. You can define a similar variable for Doctrine’s profiler and attach it to the connection in the same place where you created it. In that case you need to override the getProfiler() method in the plugin class to return that global variable.

One thing that is really missing in Joomla debug plugin, is the support of writing output to a log file. For example in some cases I don’t want to display debug information on the page (although I can hide it with CSS I guess) or I want to monitor it in the background and review later. So I added logging support to my profiler plugin. It has support for log4php and also for Joomla’s own logging system, in addition to standard output to the page. You can configure it under plugin settings. You can download it from here.

• Part 1 – Defining component structure
• Part 2 – Creating views for the front-end
• Part 3 – Creating views for the back-end

Now it’s time to add some back-end administration functionality to our little component. In general it is mostly same as in front-end: controllers, views, models. Backend lists also include functionality for sorting and paging so I created a list helper for that to decouple it from the rest of the code.

I created two separate controllers for movies and genres, because controllers have more tasks in the back-end and it’s not very good idea to stuff all that together in a single controller. Each controller contains methods for viewing, editing and saving. When updating database, I’ll use transactions to maintain consistency. Doctrine uses application level transactions and it also supports nesting. So it’s one more plus point for using Doctrine over Joomla DB layer. As far as I know, Joomla default DB layer doesn’t support transactions at all. Here is part of the movies controller:

public function viewMovies()
{
	$listHelper = new ListHelper($this, 'title');

	// Doctrine extracts JosMvMovie objects with all genres it has from this query
	$query = Doctrine_Query::create()
			->select('m.*, g.*')
			->from('JosMvMovie m')
			->leftJoin('m.Genres g')
			->addOrderBy($listHelper->getOrderBy().' '.$listHelper->getOrderByDir())
			->offset($listHelper->getRowIndex())
			->limit($listHelper->getPageSize());
	$items = $query->execute();
	$totalCount = Doctrine_Query::create()
				->select('count(m.movie_id) as count')
				->from('JosMvMovie m')
				->execute();

	$view = $this->getView('MoviesList');
	$view->setViewHelper($listHelper);
	$view->setItems($items);
	$view->setItemsTotalCount($totalCount[0]->count);
	$view->display();
}

public function editMovie()
{
	$movie = $this->getItem();
	// Used by the genres selectbox
	$genres = Doctrine_Query::create()->from('JosMvGenre')->addOrderBy('name ASC')->execute();

	$view = $this->getView('MovieDetails');
	$view->setViewHelper(new ViewHelper($this));
	$view->setLayout('form');
	$view->setGenres($genres->getData());
	$view->setItem($movie);
	$view->display();
}

public function saveMovie()
{
	Doctrine_Manager::connection()->beginTransaction();
	try
	{
		$movie = $this->getItem();
		$movie->title = JRequest::getString('title');
		$movie->imdb_link = JRequest::getString('imdb_link');
		$movie->plot = JRequest::getString('plot');
		$this->updateGenres($movie);
		$movie->save();
		Doctrine_Manager::connection()->commit();

		$this->getApplication()->enqueueMessage(JText::_('Saved'));
		$this->redirectAfterSave($movie);
	}
	catch(Exception $ex)
	{
		JError::raiseWarning(0, JText::_('Save failed').': '.$ex->getMessage());
		Doctrine_Manager::connection()->rollback();
	}
}

ListHelper you saw, is basically a wrapper for using applications state with some magic strings. One of it’s methods looks like this:

public function getOrderBy()
{
	return $this->getController()->getApplication()->getUserStateFromRequest(
		$this->getController()->getName().'.list.filter_order', 'filter_order', $this->_defaultOrderBy, 'cmd');
}

I also created a bunch of base classes for controllers and views, because I don’t like repetitive code that Joomla has all over the place. And in general it is a good idea to write your own base classes between your concrete classes and framework base classes when you have more than a few views. It‘s a good place to put component wide logic that is applied to all views. I have also encapsulated usage of Joomla global variables like $mainframe (which is actually of type JApplication) and $option into base classes. I tried to avoid usage of magic strings and global variables by encapsulating them into methods/properties, making the usage more explicit.

Views are as lightweight as they were in the front-end. They contain only setup of the JToolBar and setters for data which are used by the controllers, nothing more.

<form method="post" name="adminForm" id="adminForm">
<div class="col width-100">
<fieldset class="adminform">
	<legend><?php echo JText::_('Details'); ?></legend>
	<table class="admintable">
	<tr>
		<td width="100" align="right" class="key">
			<label for="name"><?php echo JText::_('Name'); ?>:</label>
		</td>
		<td>
			<input class="text_area" type="text" name="name" id="name" size="32" maxlength="250" value="<?php echo $this->item->name; ?>" />
		</td>
	</tr>
	</table>
</fieldset>
</div>
<?php echo $this->createMetaInputs(); ?>
</form>

In my view base class I have a method createMetaInputs() to create hidden fields, common to all views in Joomla. I also created helper method createHeaderLink(..) used by the list views. DRY.

I think it’s pretty much it. Final result looks much more cleaner and simpler than by using ‘”Joomla’s way”. Final component source code can be downloaded here.

Conclusion

Joomla don’t dictate how you should build your component. This is in some way bad (when looking the source code of the most of the components built by the community), but it’s also good. It allows us to use different (and sometimes better) implementation of the DB layer (and why not other layers too). It also provides pretty good base functionality, you just have to use it wisely.

In my previous post I talked about how to create model for localized values. Now it’s time to show how I’m querying against that model.

I display language selection to the user. So all localizable data is preferably shown in the language the user chose. When displaying persons list, for example, person name is shown in the language the user chose or the first available localized value if the localization for chosen language was not found. I’ll use HQL queries for that, which return a projection from entity to DTO to contain only data I need. HQL allows better (and easier) optimization than using criteria and it’s easier to write.

So HQL that returns persons list where first name and last name are localized, looks like this:

select distinct new PersonDataContract(person.Id, firstName.Value, lastName.Value, person.ContactCard.Phone, person.ContactCard.Mobile,
	person.ContactCard.Email.Address, l.id, firstName.Language.id, lastName.Language.id)
from Language l, Person as person
	join person.FirstName tfname
	left outer join tfname.Localizations firstName
	join person.LastName tlname
	left outer join tlname.Localizations lastName
where l.id = :langId
	and (firstName.Language.id = l.id or (index(firstName) = 0 and l.id not in (select loc.Language.id from Localization loc where loc.Translation = tfname)))
	and (lastName.Language.id = l.id or (index(lastName) = 0 and l.id not in (select loc.Language.id from Localization loc where loc.Translation = tlname)))
order by lastName.Value, firstName.Value 

I also fetch the language id for localized values because I want to show to the user if returned value was in language he requested.

And matching SQL from NHProf:

When an object has many localizable properties then such a query could become too complex, too many joins. In that case it may be better to use different strategy for fetching objects. We can use NHibernate’s multi queries or Futures. I’ll show you similar example, but in a bit different context.

In some situations it would be better to show to user all translations for selected language in a single list. For example when there is a language specialist to translate all values from one language to another. In that case we need to load all translations (for specified context, eg. person names) with localizations. In this case I need also to use paging to limit the result set.

I use multi queries approach here. First I fetch translations for the selected context and languages list. And for each language I load localizations for translations in separate queries. So I make total of two roundtrips to the database.

 public IEnumerable<Translation> GetTranslationsBy(Type translationType, int startItemIndex, int numberOfItems)
{
	// Select IDs first
	var idQuery =
		Session.CreateQuery("select t.id from Translation t where t.class = " + translationType.FullName)
			.SetFirstResult(startItemIndex)
			.SetMaxResults(numberOfItems);
	var temporaryList = Session.CreateMultiQuery()
		.Add(idQuery)
		.Add("from Language").List();

	const string queryFormat = "from Translation t left join fetch t.Localizations l where t.class = {0} and l.Language.id = :{1} and t.id in (:translationIds)";

	var multiQuery = Session.CreateMultiQuery();
	foreach (Language language in (ArrayList)temporaryList[1])
	{
		var langParamName = string.Format("lang{0}Id", language.Id);
		var query = Session.CreateQuery(string.Format(queryFormat, translationType, langParamName))
			.SetInt32(langParamName, language.Id);
		multiQuery.Add(query);
	}
	multiQuery.SetParameterList("translationIds", (ArrayList) temporaryList[0]);

	IEnumerable<Translation> result = new List<Translation>();
	var list = multiQuery.List();
	if (list.Count > 0)
	{
		result = (IEnumerable<Translation>)((ArrayList)list[0]).ToArray(translationType);
	}
	return result;
}

repository.GetTranslationsBy(typeof (PartnerNameTranslation), 0, 10); 

With results:

The third query is similar to the first two, only for different language.

I’m using only the first result set from the second batch. Other queries are only for fetching all available localizations for translations and actual results they return, will be discarded. That way I pre-load all the values and therefore there is no need for lazy loading them which in turn would result in executing multiple queries to the database.

As I mentioned, the last approach may also be used when loading DTO’s, only in that case mapping to DTO should be done by the user (or use AutoMapper in some extent).

• Part 1 – Defining component structure
• Part 2 – Creating views for the front-end
• Part 3 – Creating views for the back-end to manage movies

In my previous post we set up initial structure for the component and generated models. Now it’s time to add some front-end views. I’ll show you how Doctrine can be used in controllers and how to create clean and simple views.

When using Joomla pattern, then most of the work is done in view’s display() method. I don’t like that. I’d like that my controller provides needed data to the view. So we need to create a base controller from which our component controllers inherit to provide an injection point for our views. View is created in controller’s display() method so we need to override that but most of the code remains same, though.

protected abstract function onViewLoaded(MvViewBase $view);

public function display($cachable=false)
{
	$document = JFactory::getDocument();

	$viewType = $document->getType();
	$viewName = ucfirst(JRequest::getCmd('view', 'topic'));
	$viewLayout	= JRequest::getCmd('layout', 'default');

	$view = $this->getView($viewName, $viewType, '', array('base_path' => $this->_basePath));

	// Set the layout
	$view->setLayout($viewLayout);

	// Display the view
	if ($cachable && $viewType != 'feed')
	{
		global $option;
		$cache = JFactory::getCache($option, 'view');
		$cache->get($view, 'display');
	}
	else
	{
		$this->onViewLoaded($view);
		$view->display();
	}
}

public function getModel($name='', $prefix='', $config=array())
{
	return false;
}

So now controller can be very simple – only load data using Doctrine and initialize the view. Views contains also only setters and getters for the data they need.

class Controller extends MvControllerBase
{
	protected function onViewLoaded(MvViewBase $view)
	{
		switch($view->getName())
		{
			case 'movies':
				$this->dataBindMoviesView($view);
				break;

			case 'movie':
				$this->dataBindMovieView($view);
				break;
		}
	}

	private function dataBindMoviesView(ViewMovies $view)
	{
		$query = Doctrine_Query::create()
				->select('m.*')
				->from('JosMvMovie m')
				->orderBy('m.title ASC');

		$movies = $query->execute();
		$view->setMovies($movies->getData());
	}

	private function dataBindMovieView(ViewMovie $view)
	{
		$movieId = JRequest::getInt('id', 0);
		$movie = Doctrine_Core::getTable('JosMvMovie')->find($movieId);

		$view->setMovie($movie);
	}
}

Doctrine (like most of the ORMs) can load related data automatically so no extra work needed for that. In our case, I can just ask genres from the movie, I don’t need to load them separately. Doctrine also allows tuning how and when relations are loaded – are they lazy-loaded or joined directly when executing query, for example. You can read more from here and here.

Because we have many-to-many relation between movies and genres, automatically generated models need some tuning. We have to specify the relation class name so that Doctrine can detect how it can fill the collections. When looking the definition of BaseJosMvMovie class, which contains mappings with the data model, then there is:

public function setUp()
{
	parent::setUp();
	$this->hasMany('JosMvMovieGenre', array(
		 'local' => 'genre_id',
		 'foreign' => 'genre_id'));
}

We need to add refClass attribute there which defines the class for many-to-many relation.

public function setUp()
{
	parent::setUp();
	$this->hasMany('JosMvGenre as Genres', array(
		 'local' => 'movie_id',
		 'foreign' => 'genre_id',
		 'refClass' => 'JosMvMovieGenre'));
}

Same kind of change needs to be done with the BaseJosMvGenre too, if you want access movies from the genre. Now we only need to add some views to display our movies. They are dead simple.

class ViewMovies extends MvViewBase
{
	private $_movies;

	public function setMovies(array $movies)
	{
		$this->_movies = $movies;
	}

	public function getMovies()
	{
		return $this->_movies;
	}
}

And HTML for that.

<table width="100%" border="1">
	<caption>Movies</caption>
<thead>
<tr>
	<th>Title</th>
	<th>IMDB</th>
	<th>Genres</th>
</tr>
</thead>
<tbody>
<?php foreach($this->getMovies() as $movie): ?>
<tr>
	<td title="<?php echo $movie->plot; ?>"><a href="<?php echo $this->getRouter()->createMovieLink($movie); ?>"><?php echo $movie->title; ?></a></td>
	<td><a href="<?php echo $movie->imdb_link; ?>" target="_blank">IMDB</a></td>
	<td><?php echo join(', ', $movie->Genres->getData()); ?></td>
</tr>
<?php endforeach; ?>
</tbody>
</table>

So this is it for now. Hope it gets you started. In my next post in the series I’ll create views for the back-end, there are things a little bit different, but not much. Full source code with both the views can be downloaded here.

• Part 1 – Defining component structure
• Part 2 – Creating views for the front-end
• Part 3 – Creating views for the back-end to manage movies

I have done plenty of development on Joomla platform. Although generally it’s pretty good CMS platform for PHP but the part of that I don’t like is it’s MVC implementation. The Model implementation feels just so weird to me. Maybe because I haven’t used to that approach it uses, because I’m more NHibernate guy from .NET space… Anyway, I decided to look for ORMs in a PHP environment and found Doctrine. I did a simple test project with that and decided to use it to develop a simple Joomla component and keep things as simple as possible.

In it’s standard implementation of MVC in Joomla, controller injects needed models to the view and view asks data directly from the models. In other cases (when there is no need for a view) controller uses models directly to accomplish some task. I don’t like that. I’d like when there is just a plain view which is filled with the data by the controller and I would loose the concept of model (in terms of Joomla) wholly. Model layer will be replaced with the Doctrine and it will be used only by the controller not views.

So lets start.

Sample component

For a sample component we use simple movies catalog. It contains models for Movie and Genre. Movie has attributes like name, plot and link to IMDB. Administrators can manage movies and genres from the back-end. From front-end users can browse movies and filter them by genre and name. This should me simple enough to start with. The purpose of this post is to demonstrate how to integrate Doctrine into Joomla, not to demonstrate capabilities of Doctrine. This is the data model:

Firstly we need to install Doctrine under the Joomla libraries folder. You can download the latest stable version (which is 1.2.1) from here. After downloading extract the contents of the lib directory under Joomla libraries/doctrine directory.

Configuring Doctrine and creating models

First we need to configure Doctrine to use Joomla database settings. Interesting thing here is the table name format. Firstly we use the Joomla configured table name prefix. Secondly we add our own table prefix because we don’t want to use all the Joomla tables (at least in this case). Here is sample code.

$componentTablePrefix = 'mv_';
$config = JFactory::getConfig();
$ormManager = Doctrine_Manager::getInstance();
$ormManager->setAttribute(Doctrine_Core::ATTR_TBLNAME_FORMAT, $config->getValue('config.dbprefix').$componentTablePrefix.'%s');
$ormManager->setAttribute(Doctrine_Core::ATTR_AUTOLOAD_TABLE_CLASSES, true);
$ormManager->setAttribute(Doctrine_Core::ATTR_VALIDATE, Doctrine_Core::VALIDATE_ALL);
$ormManager->setAttribute(Doctrine_Core::ATTR_QUOTE_IDENTIFIER, true);
$conn = Doctrine_Manager::connection(
    $config->getValue('config.dbtype').'://'.
    $config->getValue('config.user').':'.
    $config->getValue('config.password').'@'.
    $config->getValue('config.host').'/'.
    $config->getValue('config.db'), 'default');

Then we use Doctrine to generate object model from the database tables for us. Of course you can create (or update) them manually if you want, there’s a pretty good documentation on that. We use this command to do that.

$modelsPath = JPATH_SITE.DS.'administrator'.DS.'components'.DS.'com_movies'.DS.'models'.DS;
Doctrine_Core::generateModelsFromDb($modelsPath, array('default'), array('generateTableClasses' => true));
Doctrine_Core::loadModels($modelsPath);

Unfortunately Doctrine also adds table prefix to the class name. So table name jos_mv_movie is converted to class name JosMvMovie. Generally I would remove that but currently I leave it as it is.

Creating a component structure

Next we create a bootstrapper script for our component to do all the wiring for us. It will include setup for Doctrine and for our component. We use PHP spl_autoload_register function to provide functionality to load all necessary scripts automatically without the need to scatter our scripts with require statements. Because Joomla uses __autoload function, we also need to re-register Joomla autloader with spl_autoload_register. That’s because __autoload can be used only once but we need different autoload functions – for Doctrine, our component and for Joomla itself.

Note: Because of Joomla isn’t written for PHP5 specifically, you need to explicitly set JLoader::load function to static.

We will use the same bootstrapper for front-end and back-end so we need to create separate configuration options for those. Because in front-end we need to load classes that are found from the front-end directories but also classes that are defined in the back-end, whereas database configuration remains same. And when creating some integration tests you may want to connect to different database.

Here is part of the code for bootstrapper.

class MvBootstrapper
{
    private static $configurationMode;
    public static function configure(ConfigurationMode $mode)
    {
        self::$configurationMode = $mode;
        self::configureJoomla();
        self::configureMovies();
        self::configureDoctrine();
    }
    private static function getModelsPath()
    {
        $modelsPath = JPATH_SITE.DS.'administrator'.DS.'components'.DS.'com_movies'.DS.'models'.DS;
        return $modelsPath;
    }
    private static function configureJoomla()
    {
        spl_autoload_register('JLoader::load');
    }
    private static function configureDoctrine()
    {
        JLoader::register('Doctrine_Core', JPATH_SITE.DS.'libraries'.DS.'doctrine'.DS.'Doctrine'.DS.'Core.php');
        JLoader::load('Doctrine');
        spl_autoload_register('Doctrine_Core::autoload');
        $ormManager = Doctrine_Manager::getInstance();
        $ormManager->setAttribute(Doctrine_Core::ATTR_TBLNAME_FORMAT, self::$configurationMode->getTablePrefix().'%s');
        $ormManager->setAttribute(Doctrine_Core::ATTR_AUTOLOAD_TABLE_CLASSES, true);
        $ormManager->setAttribute(Doctrine_Core::ATTR_VALIDATE, Doctrine_Core::VALIDATE_ALL);
        $ormManager->setAttribute(Doctrine_Core::ATTR_QUOTE_IDENTIFIER, true);
        $conn = Doctrine_Manager::connection(
            self::$configurationMode->getDbType().'://'.
            self::$configurationMode->getDbUsername().':'.
            self::$configurationMode->getDbPassword().'@'.
            self::$configurationMode->getDbHost().'/'.
            self::$configurationMode->getDbName(), 'default');
        Doctrine_Core::loadModels(self::getModelsPath());
    }
    private static function configureMovies()
    {
        MvBootstrapper::registerPath('base', self::$configurationMode);
        MvBootstrapper::registerPath('models/generated', ConfigurationMode::BackEnd());
        spl_autoload_register('MvBootstrapper::autoload');
    }
    public static function autoload($className)
    {
        // Autoload logic
    }
    private static function registerPath($path, ConfigurationMode $mode)
    {
        // Path registration logic
    }
}

Now we need to create startup scripts for our component. It’s nothing special, just a regular component starter page. Here is the code for front-end view.

defined('_JEXEC') or die('Restricted access');
require_once (JPATH_COMPONENT_ADMINISTRATOR.DS.'bootstrapper.php');
MvBootstrapper::configure(ConfigurationMode::FrontEnd());
require_once dirname(__FILE__).DS.'controller.php';
$className = 'Controller';
$controller = new $className();
$controller->execute(JRequest::getCmd('task'));
$controller->redirect();

That’s it for now. In my next post I’ll show how to create controllers and views for the front-end. And how to use Doctrine in controllers and to pass data to a view.

You can download current version of the component from here.