Better Rules: implementing digital legislation

Media Suite Developer Ersin Buckley recently participated in a project for digitalising legislation. The working group included specialists from across government and the private sector. Media Suite loves sending our crew out for these real-world learning experiences. Here’s more about what he learned. 

The concept of digital legislation is one I have been investigating for some time.  In 2017 I was having a lot of fun experimenting with prolog, so development of expert systems seemed to be an ideal use-case for using the language. When I stumbled into the regulation as a platform (RAAP) product, it piqued my interest in how expert systems could be applied to the domain of law. RAAP proposes that implementing digital standards for law would empower people to better understand how the law impacts them. I could really get behind the social good this kind of system could provide people. On another level, I really enjoy learning about new problem domains that technology can be applied to. Law and regulation is something I was a complete beginner in.

After my initial investigation, I determined the next most important thing was to start testing ideas with policy and legal drafting experts. Fortunately the perfect opportunity came up with the service innovation lab’s Better Rules for Government design sprint. Going into the investigation I hoped to learn about how legislation is developed, and how the process and tools could be used to better support digital service implementation.

The core team involved people from MBIE, IRD, PCO and DIA. We had representatives from the business rules analysts, to law experts and service designers. The input I provided was software expertise and experience implementing digital services from legislation.  Outside of the core team we attracted a diverse tribe of stakeholders who joined us for a demonstration of the work done at the end of each week.

One of the areas where I learned the most was about the actual process and people involved in making a new rule. I was excited to find many similarities between legal drafting and writing code. During the sprint we experimented with co-creation of a legal draft with a cross functional team involving policy, law, service design and software people. We looked at the Rates Rebate Act to create a new draft that meets the same intent. The theory was that a co-creation team would produce a solution that is well set up for digital service delivery. I found the process of co-creation to have many of the same positive features that Agile software delivery has.

In the software world we refactor code to reduce technical debt and make a maintainable system. Within government the same thing is done with law. When a rule is implemented, there is an opportunity to improve other parts of the act. Continuous improvement happens to make the law more maintainable and improve quality. These changes do not alter the intent of the law, but they make it easier to use and understand.

When I’m writing code, despite feeling like a genius when it starts working, I can never trust that the code is ‘correct’.  At Media Suite we do code review to improve the quality of our code. Legal drafting has an analogous process where peers can test and provide suggestions for better structure of the law. This means when it gets enacted, there is less chance of it requiring amendments.

The structure of an Act has a ‘interpretation’ section. In software thinking, this reads like a description of the data-structures in a system. It includes the definitions of nouns and actions that will be in subsequent sections of the text. You can see an example of this in the interpretations section of the Rates Rebate Act

I’m constantly reminded of functions when I read a section of the Act. Each section will mention a number of terms from the interpretation section, these are the parameters to a function. A return type for the function can be thought of as the conclusion of the term. Sometimes that may be a numerical value such as “amount of money”. Other times it may be a boolean such as “the person is entitled to the rebate”. In the law, clauses may depend on other clauses, this is like a function that composes other functions to achieve your desired result.

One day during the sprint we sat down side by side to experiment with the idea of implementing legislation as code.  My weapon of choice for our legislation as code experiment was a simple jupyter notebook. I wanted to test if I could implement the rules alongside the legal draft in parallel. We delivered the digital legislation with three supporting components, pseudocode, legal text and python code. You can check out the python code on my Github here.

In the future, a common framework for law development would enable a business to import the rules into their own systems. This means that when a law has been scheduled for change, organisations will be able to test the change and model the impact of the change. An open standard for the interpretation and definition of digital legislation will enable this. The first step towards digital legislation is to form co-creation teams to implement law. This means that service delivery, software developers, law experts, policy people and subject matter experts will be working together. At Media suite, this is an area where we have some experience. We bring Agile to a lot of challenges not related to software, our business is as much about creating behaviour change in organisations as it is about delivering technical goodness.

On a personal level, I really believe that the digital legislation will bring better information to people. By implementing law that is machine consumable, there are many opportunities to build tools that can inform people about the impact of policy. In addition to this, the new style of legislation will reduce the cost of implementation and compliance. By taking the first step of creating the co-creation team, solutions will be well suited for service and software implementation.

 

Sublime tips for multi-cursor mastery

Sublime Text is a clean and simple looking text editor, but it is powerful, and has plugins for almost everything. While cleaning out some old records of the plugins and settings we commonly used, I decided to freshen them up and throw in some tips on keyboard shortcuts and multi-cursor editing. This isn’t a comparison of IDEs as I have hardly used many others, but most of them have similar features and plugins, so this may still be relevant.

Multi-line editing.

Like 2 people using a keyboard at the same time!

One of the most convenient features that (at least originally) set Sublime apart from other editors is the ability to edit multiple lines or places at the same time. Here are a some examples of when this is useful.

I very frequently need to select and change multiple instances of some text, for example renaming a variable, or changing a parameter on all references of a function. Most commonly I use cmd+d to select the next instance of selected text. In the animation below I select the first instance (with alt+shift+left), then press cmd+d to select the next instance. If you want to skip an instance you can press cmd+k, cmd+d. Once you have your selections, you can start typing to replace it, or press left or right to start typing before/after.

As an alternative to the initial selecting of the text, with your cursor beside (or inside) the word you want to select, you can press cmd+d to “expand the selection to the current word”. The benefit of this (which I just learned in writing this) is that latter presses of cmd+d will only select full-matches of the initial word. In the above example, it skips substr automatically.

Better yet, if you know you want to select all instances within the current file, you can highlight your interested text and press ctrl+cmd+g. I use this command so regularly that I have it mapped to cmd+shift+a, which is like select-all (cmd+a) except means select-all-of-these.

You can also select multiple lines at a time using a mouse. Alt+dragging selects an area that’s split into each line. This works great if the lines have the desired content at the same character positions, or if you’re wanting to only select the longer lines. Alternatively, you can select a region (by mouse or shift+arrows) then press cmd+shift+l to split the selection into lines. I use this a lot for formatting, and you quickly learn tricks for navigating around with multiple cursors (e.g. the different combinations of cmd, alt, shift, ctrl and the arrow keys). One thing to watch out for is if line-wrap is on, this can complicate navigation of multi-cursors. I have line-wrap on/off toggle mapped to alt+w for quick switching.

Another useful way of selecting text is to expand the selection(s) to the current scope using cmd+shift+space. Wherever your cursor(s) are when you press this, they expand to the enclosing scope. This could be the current word, some enclosing quotes around a string, the current function or object definition, all the way up to the whole file. In a similar vein, you can expand your selection to the block of code at the current indentation level with cmd+shift+j.

Getting around.

The command palette cmd+shift+p gives you access to all of the menu commands in an easy to search list. This includes everything from Sort Lines and Set Syntax to commands from your installed packages Git: Diff (see plugins below).

Easily set the syntax formatting of the current file.

Quickly switch to any file in the project with the Goto file menu cmd+t. This file searcher allows partial (fuzzy) matching e.g. pstscr could match public/styles/screen.scss, and intelligently shows best matches or more recently used files first. It conveniently shows a preview of each file if you tap up/down and opens the file when you hit return. This is one of the fastest ways to switch between files, alongside ctrl+tab (mentioned below).

Quickly preview and open files by name.

The Goto menu also allows you to jump to a definition within a file by typing @ to search for a variable or function definition. This is so useful I’ve mapped it directly to cmd+r. Or you can Goto a particular line number by typing a colon (:) followed by a number into the Goto menu.

Jump directly to a function definition.

Just like in the browser, to switch between tabs (open files) you can use ctrl+tab. However in Sublime it behaves slightly different, and arguably better. Instead of just progressing from left to right (or right to left with ctrl+shift+tab), it goes in order of last viewed. This means if you’re flicking repeatedly between 2 files that you’re editing, you only need to hit ctrl+tab each time, instead of ctrl+tab one way and ctrl+shift+tab back.

Projects

If you’re using Sublime, make sure you remember to set up projects. By saving a set of folders as a project I can quickly switch between them, and it retains all of the tabs I had open when I switch back. It also remembers other useful things, such as your frequently used files (useful for Goto searching) and find-in-folder history. It doesn’t, however, retain undo/redo history.

Sublime Projects remember your search settings.

Other useful commands

  • Commenting out lines of code with cmd+/
  • Moving lines up/down with cmd+ctrl+arrow
  • Duplicating the current selection with cmd+shift+d
  • Split your window into 2 columns alt+shift+2

Sublime also has a concept of bookmarks, though they’re not the easiest to use, and there are some plugins that make them a bit more friendly. I never got in the hang of using them, but one trick I use more than I’ve ever seen other’s use is cmd+z. Yes, everyone knows undo/redo, but I so often need to go looking for something elsewhere in the file that I’m editing, and instead of trying to scroll back, I just hit undo/redo and it skips me back to where I was last editing.

Copy-paste history

While this last command isn’t anything to do with Sublime (although I originally got the idea for it from an old Sublime package), it’s so vital to my everyday development that I usually preach it when I can. Copy-paste history is one of the most useful tools I’ve ever installed, not only for development but general computer use. Not only does it save your ass when you accidentally copy over top of something important in your pasteboard, once you get used to it you have a much faster way of writing code, moving info between emails, keeping track of recently used email addresses or IDs, etc. I use Flycut for mac (see configuration image), which lets me cmd+shift+v to paste the last item, or to paste something from history I just keep pressing v or use up/down keys. This also gives the added benefit that cmd+shift+v pastes without formatting, so I can copy some HTML into a Google doc as plain text.

Configuration for Flycut copy paste history tool.

Useful configuration

binary_file_patterns

This lets you ignore files or folders when using Goto (cmd+t) or find in folder (cmd+shift+f). I usually put node_modules/**, dist/** and other directories I don’t care about in this list, making Goto and search much faster and more relevant.

draw_white_space “all”

This is particularly useful when working with different projects and different standards between spaces and tabs. It may look messy or distracting initially, but you’ll soon just see it as white-space with extra information.

always_show_minimap_viewport

It’s not a big deal, but this subtle highlight of where in the minimap you’re looking is quite useful.

See my settings file for these and more.

Useful plugins/packages

Install package control

Even though most IDEs have built-in installers, and package control has been around for Sublime 2 & 3 for years, we still have to manually install it. Once installed you have a world of excellent, and basically essential, plugins available. Here are some of my must-haves.

  • editorconfig automatically formats your files to be consistent. All of our projects have a .editorconfig file at the root to tidy up the tabbing, new lines, whitespace, etc.
  • gitgutter gives a simple overview of your changed lines within a file. I’ve found this to be more reliable than the later versions of modific. Though I disable show_markers_on_untracked_file as it’s not useful to see that every single line of a file has been added.
  • git gives you handy commands like diff, blame, and even push/pull etc.
  • SublimeLinter – standard tells me when any of my JavaScript isn’t formatted the same as everyone else’s. Yay for someone else making decisions for me. You can also get SublimeLinter for most flavours of development.
  • There are also syntax highlighting and linting packages for most of your less-common formats and languages (e.g. JSX, SCSS).

While it may not give you all of the “intelligent” features like smart debugging, Sublime does fill the job of editing files pretty well. With the powerful multi-cursor editing it’s an especially good tool for those tricky times you need to manually edit CSVs, SQL, shape files or GeoJSON. Particularly when combined with the amazing regex find & replace functionality (but that could be its own blog post)! Sublime can handle files with many lines comparatively well, but it falls over very quickly with files that have very long lines.

Are there any important tricks I’ve missed? Think you can sell me to make the move to a better™ editor? Let us know in the comments below.

The Sublime animations in this article were created using a fork of Sublime’s Animation Encoder.

Cruise Guide: The Case Study

Media Suite Director George Wills recently shared our Cruise Guide project at the Department of Internal Affairs Emerging Tech Series. Thanks to Cyclone Gita, he presented remotely. Just hours before the presentation, Version 2 of the app was released.

Read on to find out more about the 25+ APIs and Open Data sets that went into this project.  

Media Suite collaborates on a number of projects with the Marlborough District Council. One of the best known public-facing projects is Cruise Guide – an app designed to collect data on recreational use in the Marlborough Sounds, while simultaneously providing a handy tool for boaties and holidaymakers.

The Background

The Marlborough District Council manages the most coastline of any council in New Zealand. The region is diverse, both sub-tropical and sub-alpine. It is extremely biodiverse, and also holds unique cultural significance for New Zealanders. The area caters to a wide range of commercial ventures, including the majority of New Zealand’s $380 million aquaculture industry. By 2025, the industry hopes to be worth $1 billion to our economy.

It’s diversity is one of its greatest assets, but also makes it a minefield to manage. The council must balance commercial ambition with recreational preservation, environmental concerns, and cultural significance. No easy feat!

The council receives a lot of information on most aspects of the region, but it is particularly hard to gather data on recreational use. For starters, many users do not live locally, making them hard to engage with. Communication is challenging.

Time and again, Keith Murray’s 1979 book, New Zealand Cruising Guide, was referenced regarding recreational use in the Sounds. Given its age, and its print media status, the information wasn’t always up to date. The council decided the best way to gather current, relevant information was to develop an app for boaties, which simultaneously collects useful information that could then be used to balance the equation.

The Technical Guff

The council collaborated with Media Suite on this challenging project.

More than 25 Open Data sets and APIs later, Cruise Guide was eventually born.

APIs and data sets were used for the following:

  • Private and club boat moorings (MDC)
  • Resource consent data and APIs for foreshore structures (MDC)
  • Special use zones like waterskiing lanes
  • Public facilities like onshore toilets and fresh water sources
  • Department of Conservation campgrounds and huts

One of the most interesting challenges was integrating nautical charts. More than a dozen Land Information New Zealand charts were “stitched together” on one tile server, which runs on the phone and works without cell coverage, ensuring boaties can access the charts from anywhere, at any time.

Our developers also got ambitious when it came to providing useful tide predictions. Originally, the idea was to grab a MetService API. But it only gave predictions for Picton. It also wouldn’t work offline. In the end, it was a data set updated annually by LINZ, in spreadsheet form, that was the answer. Using four years of tide predictions, the app uses calculated offsets for 28 secondary ports, providing accurate tidal predictions, plus or minus two minutes. It’s also linked to the bays as well. Users can search predictions four years in advance.

Using this method does come with a risk, as LINZ has not guaranteed it will continue to update this spreadsheet. However, it does currently provide the most accurate data available.

The app also allows the Harbourmaster’s Office to send push notifications to users about weather warnings or useful information. App users can also send information the other way, updating the Harbourmaster on incidents at sea or advising of broken facilities. Users later requested live webcam feeds so they could check weather conditions in advance. The council then installed those webcams around the Sounds which are now linked to the app.

The Challenges

A lot of the data was not immediately fit for purpose. Work was required to manually “mash it up” and make it user-friendly. The tide data found in a spreadsheet would’ve been easier to use in an API.

Cruise Guide aimed to provide the names of all the bays in the sounds – which meant combining all various records for the area into one. It needed a GIS analyst and significant local knowledge to compile the dataset, and even then, some bay names were disputed. The council therefore chose not to release the bay names as Open Data.

Murray’s original book provided some useful detail that was included in Cruise Guide. However, the licensing of that content took a year to finalise. While Cruise Guide was finished in 2015, it was not released until the deal closed in 2016.

The Successes

“Aside from spending quite a bit of time up in Marlborough on boats doing user research, what worked exceptionally well in the project was bringing together a really diverse set of stakeholders from different departments in the council. They all got to collaborate on the design and roadmap for the project.” – George

Media Suite is all about collaboration – both between ourselves and our clients, and within the client’s own team. Cruise Guide relied on, and succeeded thanks to the teamwork and commitment from many council departments with a stake in the outcome.

The Product Owner was the Regulatory Department, invested in the usability of the app. The Web Team provided initial wireframes, and now manage some of the content. The council IT Department provided project governance and API integration support, and the Harbourmaster worked to ensure the app would be useful for boaties.

In fact, some of the most useful people in the project were those without longstanding boating knowledge. They helped provide the feedback that made the app usable for every person in the Sounds.

Since the app’s release, the public have been providing feedback to assist in its improvement.

At the end of the day, this app provides “data collection to inform future decision making in the council.”

Media Suite is proud to have been a part of developing Cruise Guide. You can download the app for Android or iPhone, or check it out online at www.cruiseguide.co.nz.