Surface drilling Drupal Panels 3

This article contains some notes from a short reverse-engineering session targeted at Panels 6.x-3.8 and Chaos Tools 6.x-1.8. The goal was to capture some first impressions and record them for potential future exploration and understanding of that module. The motivation came from the current lack of technical documentation from the powers that be.

Before deploying the drills, I wish to share with you a few contemplative remarks about Drupal documentation. I feel disappointed by the supposedly "user-friendly" and sometimes overenthusiastic case-oriented explanations so prevalent in descriptions of Drupal ("if you want X, click on Y... how cool!"). Frankly, I find them much inferior to what I like to call model-oriented explanations ("the system consists of parts X, which have state Y and interact according to rules Z") based on strict terminology and aiming to identify intermediate causes at work. A good model capitalizes on your prior software engineering and site building experience by allowing you to generate tons of individual cases in a reasonable amount of time. On the other hand, a set of cases helps you solve only a limited number of problems quickly by imitation. A large database of cases takes much effort to collect, maintain and remember, and it does a poor job at eliminating residual risk. When you're hit with a problem not yet documented in the database and forced to somehow augment your existing cases, this task is going to take an extraordinary amount of time or require to call in heavy artillery (a consultant). In contrast, a well-constructed model takes just a little effort to acquire (but not to create!) and promises much more substantial risk reductions; model refinements do not require quantum leaps of imagination. By abstracting well, you can keep most issues out of your mind most of the time, and with a good choice of metaphors, related pieces of a solution should come together quite effortlessly. In effect, adopting a cohesive model-oriented approach allows you to deliver more consistent problem solving for your clients. Similarly, when you hire someone, they better bring this secret weapon in their arsenal rather than just Google + d.o.

Back to the main topic. A panel is a named, persistent set of specifications that govern a joint presentation of multiple pieces of content on the same page. These specs cover

• which kinds of content or even which individual pieces of content should be retrieved from the database;
• where (in which pane of a multi-pane layout) each piece of the retrieved content should appear;
• how each piece of content should be rendered;
• under what circumstances and for whom each piece of content should (dis)appear;
• (for panel pages) which URI paths should trigger displaying the specified content.

A panel is defined by a site builder. It is stored either in the database (by default) or in code (when exported, e.g. for the sake of VCS). It is then used by Drupal as a blueprint for page construction whenever its associated path is requested by the browser. The specifications that comprise a panel are further subdivided into a (non-empty) set of variants, each of which may differ from the other variants in certain respects while sharing the common base configuration to foster reuse. (However, variants are unlike displays from Views in that they all have the same rank; there is no default variant that others would inherit from and override.)

Panels are somewhat at odds with Drupal core, meaning that they seek to replace rather than just expand upon and integrate with some of the previous ideas:

• In their capability to lay out pieces of content, panels build upon—but also to an extent compete with—page templates defined by a theme. In particular, panes of a panel layout are conceptually equivalent to regions of a page template. Because a panel is placed within a region, it can be also thought of as a means for subdividing the region into multiple sub-regions (which are not known to the theme subsystem).
• In their capability to retrieve and render pieces of content, Panels developers abandon Drupal's core idea that all content should be represented as either nodes (for user-defined content) or blocks (for reusable pieces of UI). Other kinds of HTML content, not belonging to either category and possibly computed on-the-fly, can also be made available to and rendered in a panel.

To put the above and following explanations in context, consider the following screenshot of the Panels UI (with some section numbering added):

The configuration sections are subdivided into three main areas: Summary, Settings, and Variants. The information under Summary and Settings concerns the panel as a whole, while the subsections under Variants pertain to each of the defined named variants. With a few exceptions, the section labels are not self-explanatory at first glance, and the information contained within each section cannot be easily understood separately from the other sections. In this sense, the UI is non-intuitive. It won't teach what can be done with panels in a learning-by-doing approach. It will cause concern with regard to unexplained settings and overly abstract terminology. The module author's line of defense is that you don't have to understand every piece of the UI to successfully use panels for simple layout scenarios. While true, the suggested adopt-now-explore-later approach also conveys an ominous message: if the documentation is not ever written—and if the adoption or future development of the module suffers—you may find yourself in a bad position by having placed bets on it. (It is not much consolation for new users that the friendly competition doesn't fare much better with regard to making their alternative comprehensible and that the Panels module is well-entrenched with a significant user base.) The question is: can we in the meanwhile remedy this situation by sacrificing some time on the altar of open source and simply peeking into Panels code? Or is the code as poorly done as the documentation? The rest of the article is my first shot at answers (for the impatient: yes and no, respectively). I start by exploring some of the more confusing—while at the same time important—sections of the Panels UI and conclude with a brief foray into code design and implementation issues.

It turns out that the key to understanding Panels configuration are the concepts of "contexts" and "content". These technical terms are not at all related to the Context module (best erase it out of your mind while studying Panels), only weakly related to the content types of the CCK module, and they might not match any colloquial meaning taken from a dictionary. Such is the nature of terms unleashed on the unsuspecting world at night by well-meaning software developers. Let's focus on "contexts" first. In Panels terminology, the term context refers to a specification of a piece of (renderable) content that should be loaded and become known under a chosen id within the panel variant before it is output as HTML. For example, if your panel should display three nodes next to each other, it would obviously first have to load them, which you would specify by adding three contexts of type node to some panel variant. The Contexts section (2) lets you directly specify the pieces of content that should be loaded; for example specify a node by its nid or title. Alternatively, you could parametrize the panel to load different pieces of content depending on what has been provided through the URL. To achieve this, you would include placeholders in the panel's path (as shown on the screenshot) and then use the Arguments section (1) to associate each placeholder with a context type (e.g. specify that the first placeholder called %node1 really refers to a node ID rather than a taxonomy term ID; at request handling time, the URL is first taken apart by Drupal's menu subsystem and for each placeholder all modules, including Panels, are consulted – see _menu_load_objects in includes/menu.inc for details). Finally, by using the Relationships area of the Contexts section (2) you could also specify a context that loads a piece of content related to another, previously specified, piece of content (say, load the user who is an author of the second node loaded by the node context). Based on the preceding description, a better term for "contexts" might have been "content loaders"; alas, the release night is over, and it is too late for changes now.

Now let's turn to content. In Panels terminology it refers to "something that can be placed (and somehow rendered in) a pane" through the Content section (3). Unsurprisingly, the pieces of content loaded by contexts as well as their individual parts (such as CCK fields of nodes; or the user picture from a profile) fall into this category. For this reason you are generally well-advised to first define contexts before even turning to the Content section. However, other pieces of content (such as blocks or HTML snippets) are also available without the need (or possibility) to obtain them through a context. The advantage of loading panel content through contexts is that the attributes of the loaded objects then become available within all panes as variables that can be used in string substitutions (e.g. %node1:title or %user1:name, where the prefix equals context id). For example, having established the three node contexts and one user context as mentioned above, you could add a HTML snippet (aka "custom content") with the node titles and the user name to a pane.

Having acquired a basic understanding of the context/content pair of concepts and experimented a bit with the Panels UI, here are the next few natural followup questions that come to mind:

• Where do all the selectable context and content types in the Panels UI (and their configuration forms) come from?
• How does the Panels module know to offer "Node", "Taxonomy term" and "User" contexts and how do you know what data exactly they load?
• What the heck is a "Node add form" context supposed to load and how is it different from "Node"?
• What determines which items appear on the "Miscellaneous" tab when adding content as opposed to the "Widgets" tab?
• What if you want to add something that is not at all listed in Context or Content sections?
• How exactly is the Relationships combo populated?
• Should we degrade ourselves to click monkeys and figure out all of the above by trial and error?

I won't even come close to providing detailed (boring?) answers to all of the above questions and will instead address the last (somewhat rhetorical) one by providing a little push in the right direction. As announced before, the solution is sought (and found) by examining the code. It turns out that most of the items and forms you come across while interacting with the Panels UI are not hard-coded anywhere in the Panels module, but rather contributed dynamically by code. "I know – panel hooks!" might be your first guess, but unfortunately it's not that simple. There is a lot more indirection in the Panels code than meets the eye and a more refined mechanism than good old Drupal hooks is at play. Furthermore, it turns out that neither contexts nor content types are designed to be panel-specific. In fact, most of them are not even implemented in the Panels module, but rather belong to the Chaos Tools module, upon which the Panels module depends. In general it's safe to say that the indirection and abstraction sauce is good for flexibility and extensibility and also a definitive poison for code comprehension. The case of Panels is no exception, and so I guess that the author might be playing with fire to some extent ("we have this wonderfully flexible super-duper API for every conceivable task, which unfortunately noone besides us can wrap their head around). I'm glad to report that I have seen much worse and that the code design and inline documentation are definitely more solid than what you see in "advanced help" today. In short, you can wrap your head around it.

To navigate around Panels code—and to see how the administrative UI is built and what each selectable item stands for—you have to first grasp the much more general concept of CTools Plugins (reified hooks). This part of the Chaos Tools module is an example of the Strategy pattern at work. It aims to facilitate a division of labor between module developers and site builders. A plugin instance is an exchangeable persistent object responsible for performing a set of tasks in its own particular way. Plugins are typically selectable (and so instantiated) through the UI of those Drupal modules that depend upon them (like Panels or Views). They may also contain callbacks that construct an administrative form-based UI to allow site builders detailed configuration of each plugin instance. Alternatively, plugins may be organized hierarchically (in a type:subtype or parent:child fashion), for the benefit of a more structured selection UI. The exact set of responsibilities for a plugin type is usually specified by the module developer who invokes plugins of that type from her module. Particular implementations of those responsibilities are then provided as plugins by (potentially) different developers. A plugin is usually implemented in a single .inc file. Plugin names must be unique within a plugin type, but do not need to be unique across types. Plugins are used by modules or by each other. The CTools library acts as an intermediary and a directory service for publishing, locating and invoking plugins. This terminology and the discussed relationships are illustrated by the following examples:

Plugin type	Responsibility of plugin type	Plugin	Plugin functionality	Plugin implemented in	Plugin used by	Plugin instance (examples)
contexts	Provide renderable content for a panel.	node	Load and provide data from a node specified by nid or title.	ctools/plugins/contexts/node.inc	panels.module (indirectly, through its own plugin panels_context of type task_handlers)	node1 and node2 in Variant1 of panelpage1
"	"	user	Load and provide data from a user profile by uid or name.	ctools/plugins/contexts/user.inc	panels.module (indirectly, through its own plugin panels_context of type task_handlers)	user1 in Variant1 of panelpage1
content_types	Render available content within an individual panel pane.	node	Render available node content.	ctools/plugins/content_types/node.inc	panels.module (indirectly, through its own plugin standard of type display_renderers)	"Node 1: ID" content in left pane and "Node 2: ID" content in right pane of Variant1 of panelpage1
"	"	user	Render a picture from user profile.	ctools/plugins/content_types/user_context/user_picture.inc	panels.module (indirectly, through its own plugin standard of type display_renderers)	"Node1 author" user picture in Variant1 of panelpage1

The required naming conventions and steps for defining plugins (and plugin types) and invoking plugin instances are documented in "advanced help" shipped with the CTools module, which you are advised to read carefully. The CTools library is used pervasively and consistently throughout Earl Miles's (aka merlinofchaos) modules (and also in some others). Thus, if we view Drupal as a domain specific language, CTools should count as a potentially prominent dialect. Becoming familiar with it will not only help you understand Panels, but also Views and potentially future of Drupal core, if it should spread into more mainstream use.

Note: while looking at the CTools module, I also found that it supports third-party module developers in evolving and versioning their own modules' APIs, including catering to different client modules using different versions of an API at the same time. The term API here refers to the set of hooks which the developed module invokes (i.e. the hooks provided [not the hooks implemented or library functions offered!] by that module). When CTools versioning support is utilized by a module, hook implementors must partition their code into multiple files, each of which corresponds to a particular version of the API. When the hook provider is about to invoke a (particular version of) a hook, it can query CTools for all matching implementations, while excluding any non-matching (e.g. too old or too new) implementations. This mechanism, documented in "e;advanced help", appears to be almost unused, and probably will remain so due to its confusing official description. Nevertheless, it is interesting to see an idea of this sort in place, given Drupal's track record of neglect for backwards-capability (at least between major versions).