When I first got into Node.js I quickly learned about Gulp. Gulp is a task runner that favors code over configuration. This differs from something like MSBuild that we use in the Microsoft world for Visual Studio that favors a configuration over code defining tasks using XML files. What is Gulp? In a nutshell it’s a task runner commonly used in a build system that where you use JavaScript to glue a myriad of plugins to create tasks, all written in JavaScript. These tasks do things for you like concatenating all your CSS files together and then uglifying / minifying them or do some static analysis and style checks on your code or running tests or any number of other things.
Why do I like it so much? I love how the power and flexibility of the Gulp framework and ecosystem enables me to automate mundane tasks and focus on the real work in the project. For example, I can fire up a task that will vet my code for style & formatting, compile all the SCSS files to CSS, all my TypeScript to JavaScript, run all my tests dynamically update the HTML files with all the CSS & JS files in my project (so I don’t have to remember to manually add them) and start the web server. This same task also them watches for changes in any file in the source and if it sees it, it will automatically repeat the entire process above, letting me focus on the work. I usually have a console / terminal window up to the side so I can see if any red pops up when the tests are running.
But all is not great in Gulp land. There are two challenges that you come across when working with Gulp. All these tasks you write generally live in one file, the gulpfile.js that lives at the root of your source. Sometimes you only have a few tasks in your project so it’s not very long, like this one in one project I have. This isn’t too bad, but others can get real long, like this example. The challenge in these long files is the same you have with any other long file:
- Maintenance: When I have a handful of tasks in my build workflow, it quickly gets tedious to scroll through the gulpfile.js to find what I’m looking for. Granted I only do this when I’m changing or fixing my build workflow, but during those times, it’s a pain.
- Reuse: While there are plenty of Gulp plugins, you always incorporate them into your tasks and it’s fairly common for someone to reuse the tasks they create across projects. If they are in one big file with external dependencies, that’s going to make sharing them across projects harder.
TL;DR
So I came up with a solution that’s working great for me. Instead of one big gulpfile.js, I have a generic one that dynamically loads tasks from a separate folder. This way each task lives in it’s own file that is easily reused. In addition, I also use the plugin gulp-help that replaces the Gulp task creator to allow me to set a description, options and aliases for a task for a better help experience.
Gulp Tasks
You can see an example of the project in this repo on GitHub: blog-gulpts-dynaload.
Project Structure & Conventions
I use TypeScript for all my JavaScript development as I find it makes me more productive. I even use it for my Gulp work. The downside of this is that I first have to run tsc -p ./ on the root of my project to compile everything to JavaScript as I need the gulpfile.js and the things that makeup my Gulp build workflow. I only have to run this once or when I make changes to the Gulp files… the rest of the time I have a task that compiles all the server-side TypeScript (for Node.js) and client-side TypeScript to JavaScript. As such, all the code you’ll see in this post is TypeScript… specifically I am using the latest version at the time of writing: v1.6.2. With that being said, you don’t have to use TypeScript for this process, I just prefer it. Pure JavaScript will work fine with this.
Let me first show you what the project looks like. I’ve stripped some files out for simplicity as they didn’t matter here, but here’s what it looks like:
.
├── build
│ ├── build.d.ts
│ ├── gulp
│ │ ├── BaseGulpTask.ts
│ │ ├── config.ts
│ │ ├── gulp.d.ts
│ │ ├── utils.ts
│ │ └── tasks
│ │ └── ...
│ └── typings
│ └── ...
├── src
│ ├── ...
│ └── server
│ ├── controllers
│ │ └── ...
│ └── server.ts
├── gulpfile.ts
├── package.json
├── tsconfig.json
├── tsd.json
└── tslint.json
Working from the bottom-up, you’ll first see the typical files you’d see in many Node & TypeScript projects like package.json, tsconfig.json, tsd.json & tslint.json. The gulpfile.ts is where where it should be, in the root of the project… I’ll explain this in more detail in a moment.
The src folder is where my actual project codebase is, both the server-side and client-side components.
Next up is the build folder. This contains a shared TypeScript type definition file, build.d.ts, that defines a bunch of interfaces for types I use across the build process. This folder also contains a typings folder which is dynamically created and populated when you run npm install as it uses the popular tsd utility for downloading cached type definitions referenced in tsd.json (you won’t see this folder in the source as it is dynamically created).
What you’re really going to be interested in is the gulp folder where I have a few files like configuration and utility stuff as well as a base class that all my Gulp tasks inherit from. Each task resides in a separate file within the tasks folder.
Now that you see how a project is structured, let’s see how this dynamic task loading really works.
Gulpfile
Take a look at the gulpfile.ts. It’s only job is do the following:
Load all JavaScript files in a specific directory
For each file, create an array of objects where each object has two properties:
- name of the task, which is the filename less the extension
- actual task object from the file, dynamically loaded using the
require('')method
For all objects in the array, register a task using the properties from the loaded file
What’s nice about this is that this file is the exact same for every project. Now let’s see how the tasks look…
Gulp Tasks
I start with an abstract class that all tasks inherit from: BaseGulpTask.ts. This contains all the default properties like an empty description, empty array of dependent tasks to call before this task is called, empty options and aliases (love having aliases for gulp tasks).
And then I have tasks. Each task is in it’s own file. The name of the file dictates the name of the gulp task. Each file has a class named GulpTask that extends the BaseGulpTask.ts. You then override anything you want to change like the description, dependent tasks that should be run before this task, options and aliases. Finally, implement a class constructor that is the actual task. You write it the same way you’d write any other gulp task… there’s nothing special about this aside from how you structure the class.