A quick Decoupled Blocks update

Submitted by mrjmd on Tue, 05/31/2016 - 08:06

As some of you may know, my pet project at the moment is a Drupal 8 module called Decoupled Blocks.

It is based on the progressive decoupling model I helped implement for certain major meteorological sites, and allows javascript developers to write pieces of decoupled functionality without knowing Drupal API's, which are then converted into blocks for placement on pages.

The module was officially announced (see the link above) the Monday of DrupalCon, and I spoke about it in my session. It also received mentions in the Angular and Ember sessions, because one of its key features is that it aims to be javascript framework agnostic.

In the time since its announcement, there has been a lot of help from the community and a lot of big changes have already landed:

  • We have an official Alpha release! As of this morning, the sandbox on Drupal.org has been upgraded to an official project and a first release has been cut!
  • Much better resolution of component paths.
  • We moved our library definition for Angular 2 to reference node_modules and added a package.json file.
  • Basic configuration page to turn on/off Angular 2 development mode.
  • Updated Angular 2 implementation from beta to RC1. This was a major undertaking, since both long form bootstrap and component load with custom selector have changed, core parts of our architecture.
  • In your component's info.yml file, you can now use add_js and add_css to create dynamic Drupal libraries with any additional js or css assets needed.

In addition, significant progress has been made on some other key pieces of functionality, so coming next on the roadmap are:

  • Gulp process to automatically compile Typescript to Javascript when developing ng2 components.
  • Bundling process to reduce requests.
  • First version of our Ember sub-module should be out this week.
  • Improvements to React sub-module.
  • Revisiting the issue of component-specific node dependencies.

There is a lot of work still to be done, and plenty of places for interested parties to jump in and help. I'll continue to update here frequently, but if you'd like to get involved in the development / testing, please do reach out!

Angular Universal and PHP, first adventures in V8Js

Submitted by mrjmd on Fri, 05/20/2016 - 22:57

There have been multiple rumblings in the last few months surrounding Drupal and Angular 2 integration, as well as more generally PHP and Angular Universal integration. While some discovery work and even some sprinting around these efforts have occurred, as of yet there is not a whole lot to showcase. But let's have a look at what has been done so far, and where these efforts are attempting to go.

Back in February of 2016 there was a joint effort between the Angular team and some prominent Drupalists to make twig (the template engine used in Drupal 8) work seamlessly with Angular 2. This largely boiled down to using twig.js and a six line patch to Angular 2 in TemplateNormalizer that brought the whole thing together:

var _isTwig = template.templateUrl.endsWith('.twig');
if(_isTwig) {
  templateContent = twig({
    data: templateContent
  }).render(new directiveType.runtime());

There is some pretty cool potential that this points us towards. I mean, being able to use a regular twig template that ships with Drupal as your template in an Angular 2 component is an awesome notion. It means a Drupal 8 themer can write their templates just as they are used to for consumption by a progressively decoupled Angular 2 application.

But ultimately this method has some major drawbacks as well. Primarily, since it is leveraging twig.js, this implementation is moving work to the client side (processing twig templates) that is already being done server side by Drupal. Processing templates isn't really something that should be done client side, and even Angular 2 itself is moving toward an offline template compiling that was announced a couple of weeks ago at ngconf (it's not quite ready yet, though I've played with it a little and it is exciting). I'll come back to offline compiling at a later date.

There was also some parallel work done to make a custom Twig decorator for use inside Angular 2 apps. This too, has some interesting potential, but without a node.js backend to at least accompany your Drupal stack, it still keeps too much templating work in the client to be desirable from a performance standpoint. To be clear, all of this was done as purely proof of concept work, so in terms of planting signposts for us to know where to head next, I think these exercises have been a success.

What would be ideal is direct integration with Angular Universal, which is Angular 2's isomorphic initiative. This would allow us to move not only template processing, but where desirable full component rendering to the server side, so as much initial state work as possible can be done before handoff to the client.

Repeatedly in my discussions with the Angular and Drupal folks working on these topics, another possibility has been raised: V8Js. What is it exactly, why do extremely intelligent people keep bringing it up in these discussions, and why don't any of them seem to know anything more about it than the fact that it exists?

V8Js, to quote directly from the link above, is a PHP extension that "allows you to execute Javascript code... from PHP." It does this using V8, Google's open source JavaScript engine. It is a strange idea, and the first time I looked at the example code associated with it I was left somewhere between curious and offended:

$v8 = new V8Js();

/* basic.js */
$JS = <<< EOT
len = print('Hello' + ' ' + 'World!' + "\\n");

try {
  var_dump($v8--->executeString($JS, 'basic.js'));
} catch (V8JsException $e) {

Yes, you're reading that correctly - it's PHP running a basic javascript print and dumping the results. Is it an abomination or a stroke of genius? Well, frankly I haven't entirely made up my mind yet, but there's no doubt that people have been doing interesting things with it like running server-side react components using PHP instead of node.js. I want to play with it myself and see just what could be done here. After a couple of failed attempts and with the help of a dev ops colleague, I finally got the extension running using the following method:

$ sudo add-apt-repository ppa:pinepain/libv8-5.2
$ sudo apt-get update
$ sudo apt-get install libv8-5.2
$ sudo pecl install v8js

Then by adding a line to the end of my php.ini, "extension = v8js.so", and restarting apache, I was ready to test it out.

I'm not entirely sure where this is headed, but there is a working group forming now to continue this effort, including Jeff Whelpley and PatrickJS. Our first meeting is next week. My plan of attack is to keep working with V8Js and studying the existing Java integration proof of concept, which also leverages V8 using J2V8. This should help me understand the integration points with Angular Universal.

All of this work is being done in parallel to my ongoing work on the Decoupled Blocks module, which would be the main direct Drupal implementation I'd like to incorporate these efforts into. But obviously this type of integration could benefit all kinds of PHP / Angular 2 applications. In the meantime, as showcased in my presentation last week at DrupalCon New Orleans, as well as the Angular 2 session, right now you can use Decoupled Blocks module to drop Angular 2 components onto your Drupal 8 pages.

I'll post a follow up on integration progress soon. Until then, from javascript executed server side by PHP in Drupal, "hello world!"

V8Js hello world


mrjmd.com Under the hood (first adventures in Docker)

Submitted by mrjmd on Fri, 05/20/2016 - 08:58

Maybe I'm a little behind on this one, but I had never played with Docker directly before until this week. Coming out of DrupalCon New Orleans, I am still in "learn all the things at once" mode, and knew I was planning on spinning up my own site again, but wanted to try to do it with this (not so) new tool. I attended a BOF on Docker at the Con, but was in over my head and knew the only way I would really learn the basics was to dive in and give it a try myself.

So, I spun up a new droplet on Digital Ocean, shelled into it, and got to work. Here's the basic path I followed, which came largely from this article:

$ wget -qO- https://get.docker.com/ | sh
$ docker pull mysql
$ docker pull drupal:8.1.1
$ docker run --name mysql -e MYSQL_ROOT_PASSWORD=nottellingyou -e MYSQL_DATABASE=d8 -e MYSQL_USER=d8_user MYSQL_PASSWORD=stillnottelling -d mysql
$ cd ~ && mkdir modules
$ docker run -v /$HOME/modules:/var/www/html/modules -p 80:80 --name drupal8 --link mysql:mysql -d drupal

That's it! Then I navigated over to mrjmd.com and installed Drupal 8 as usual. Now let's go through all of this in a bit more detail and see what I've done.

$ wget -qO- https://get.docker.com/ | sh

This first command just downloads and installs docker. As mentioned in the article above, you can verify it has worked properly afterwards by running the command:

$ docker -D info

This will show you all the basic information about your docker install, including how many images and containers you have (zero right after running this command, or two after running all of the above).

$ docker pull mysql

This pulls down the mysql image from the official docker image hub here. Click through to read more about it.

$ docker pull drupal:8.1.1

Similarly this pulls the official drupal image from docker hub, and specifies that I want version 8.1.1. The Drupal image currently includes Apache 2.4.10 and PHP 7.06 by default, so I'm finally running Drupal 8 on PHP 7 also!

So, now I've got both of these images sitting on my remote box, but they're not actually doing anything. You can think of Docker images like snapshots in Vagrant terms, they are system images that contain a baseline setup to get you going, but aren't actually doing anything until you tell them to, which is our next step.

$ docker run --name mysql -e MYSQL_ROOT_PASSWORD=nottellingyou -e MYSQL_DATABASE=d8 -e MYSQL_USER=d8_user MYSQL_PASSWORD=stillnottelling -d mysql

There's a lot going on here, but it's not very complicated. We are starting up our mysql image (creating our first container). We've named the container simply "mysql", since I'll only be running the one here, but you can name it whatever you want. Then we're feeding in a variety of environmental variables, including our root password, a database name we'd like created immediately, and a username and password to associate with the new database. All you really need to feed in at this point is the root password, as you can always connect to your MySQL command line later and run commands directly using the following:

$ docker run -it --link mysql:mysql --rm mysql sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'

For brevity I've just used that single command to do all the work instead. Moving on.

$ cd ~ && mkdir modules

It should be obvious what's happening here, I'm simply create a new directory in my user's folder called "modules". The motivation is probably obvious as well, but I neglected this part the first time I installed my site and it took me a minute to figure out. Because docker containers are self-contained, once I originally had my site installed I couldn't find any obvious way to add on new modules. Once I ran the drupal image I could run bash commands inside of it by using the following command:

$ docker exec -it <mycontainer> bash

Where <mycontainer> is the container id for drupal, which you can find using:

$ docker ps -a

[As a quick aside, the "-a" flag above was super helpful to me at one point, because my MySQL container crashed the first time I tried to install and I couldn't figure out what was going on... the container didn't show up at all when I used "docker ps". If this happens the "-a" flag will show you all containers that exist, and once I saw mysql was stopped I just ran "docker mysql start" and was back on my way.]

However, I wanted to be able to download modules directly onto the remote box and mount them into a volume that would be used by my container, hence I needed a new directory to mount. Which brings us to:

$ docker run -v /$HOME/modules:/var/www/html/modules -p 80:80 --name drupal8 --link mysql:mysql -d drupal

Here, first and foremost, I am running the Drupal image and turning it into an active container. Further I'm mounting the directory I just created as a volume and linking it to the appropriate directory inside the drupal image, setting port forwarding, naming the container, and linking it directly to the mysql container we already have running. Boom!

After that navigating to mrjmd.com redirected me to the install page, and I installed Drupal normally, using the database credentials I specified when running the mysql image. The only caveat was opening the advanced fieldset under database settings and changing "localhost" to "mysql". Now I'm up and running, with the ability to download and add contrib modules easily. Excuse me while I go install pathauto finally.

Next steps will be getting some php extensions added, check back soon for my progress on that.

First adventures in GraphQL

Submitted by mrjmd on Wed, 05/18/2016 - 07:10

Last week at DrupalCon New Orleans I attended the session by Sebastian Siemssen on GraphQL and Drupal (recording here). GraphQL first hit my radar when I came across a talk at last year's NYC Camp about it by Preston So, and I've been excited about its potential and mentioning it when I speak ever since. So I was eager to finally see it in action working with Drupal, and even more pumped to get it up and running on my own machine so I could play with it myself.

Sebastian's session was excellent and covered a lot of the exciting features already in the module as well as where the gaps were, but unfortunately by the end I still didn't fully understand how to make Drupal and GraphQL work together. I could see it working on his machine but had no idea how to make it work on mine. He emphasized though that he was looking for help and was planning a BOF for the next day to dive into the details. I gave him my card after the session was over and he let me know when and where the ad-hoc BOF was going to be.

There were four of us in the BOF, which we held in the sprint room since all the actual BOF rooms were taken. We announced "GraphQL BOF" beforehand and one other guy came over, asking what GraphQL was and what kinds of graphs you could make with it. We explained that's not quite how it worked.

For those of you who don't know, and can't be bothered to watch the excellent videos above, I'll try to be brief - GraphQL is a query specification created by Facebook, one that allows declarative querying of a single endpoint. If you've worked with decoupled sites lately you know the typical way backend data is retrieved by the client side, through RESTful endpoints usually formatted as JSON. This approach can work really well under the right circumstances, when the back and front end teams are communicating effectively (or are the same person), endpoints can be created exactly as and when needed, with the structure that the client side application requires.

However, there are a few problems that can arise from this approach:

  • On large sites, you can end up with a LOT of endpoints. Unless they are self-documenting (that's a joke) or the front end team is familiar with how to find and use them, having dozens or even hundreds of different endpoints with variations on types and structures of data can quickly become very confusing.
  • When a new piece of front end functionality is needed, a new endpoint may be needed, necessitating two different teams to do work and creating unnecessary cycles. Or, even worse, a front end developer may just decide to use an existing endpoint that already provides more than enough data and iterate through it for their specific needs, or use a combination of endpoints to get the data they need leading to multiple requests.

GraphQL  solves these issues, by letting the client request data, specifying the exact type and format they desire as part of the request itself. Then GraphQL simply fills in the request and sends it back. This means that a single endpoint can be powerful enough to serve all of the data of your backend site out.

Before I show an example though, let's go through how to spin up a GraphQL server on your own local Drupal 8 box and try it out yourself. First things first, you will need a Drupal 8 site up and running, as well as composer installed. Once you've got that ready, it is really only a couple of commands that you need to run. But they must be run from the docroot of your Drupal site.

// This will add a new repository to your composer config.
$ composer config repositories.drupal composer https://packagist.drupal-composer.org
// Then install the module as well as the PHP library all at once.
$ composer require drupal/graphql 8.x-2.x-dev

After that, turn on the GraphQL module in Drupal as usual, and you should be ready to rock. Navigate over to /graphql/explorer to test out your GraphQL server with a basic query (query is on the left, response on the right):

Basic GraphQL Query

I'll be working with Sebastian to document out more of what is possible in the coming weeks... but to take the next step towards using it in your actual applications, I recommend experimenting with queries in the explorer like above, and then opening the network tab of your debugger and looking at what the actual request sent to the endpoint looks like.

Hello world

Submitted by mrjmd on Tue, 05/17/2016 - 09:19

mrjmd.com is alive!

I just installed this site using docker (mysql and drupal's official containers), which I have never used before. Still a lot to learn about docker but it was past time for me to dive in. I just got back from DrupalCon New Orleans and had an exciting week. I have a lot to write about, including:

  • Decoupled Blocks module
  • GraphQL
  • Angular 2 / React / Ember
  • V8JS
  • Component driven theme rendering
  • Much more!

This site is going to serve as a blog/thought repository and also a sandbox, so I do plan on installing experimental things and showing off what works (and what doesn't). Decoupled Blocks will be here soon, but for now check out the blog post on Mediacurrent's site announcing it: https://www.mediacurrent.com/blog/introducing-decoupled-blocks-module-d…