Information Architecture (IA) Change Process



How to make data changes clear

Why?

tweet

Why?

  • Larger changes to a DB require understanding & agreement across
    • Product Management
    • Data (BI & DI)
    • Software
    • & often Operations / Stakeholders
  • Teams need communication tools to show data change & intents
  • A change process that allows for incremental change during discussion
  • As DB complexity grows it’s more difficult to make a correct schema.
    • Particularly taking into account multiple aspects of the business
  • Many DBs have legacy tables & fields that can further confuse conversations around data
  • As DBs grow teams need an easy way to focus in on only the models under discussion

How?

  • Integrated into development & PR flow
  • Supports Async discussion & feedback
  • Both visual & textual diffs
    • DOT <-> PNG
  • Faster to express than full code change
    • DB changes don’t actually need to be applied for visualization
    • Migrations don’t need to be written or run
  • BI & PMs can edit DOT files to express intents
  • Fast iteration with images can be updated during live workgroup discussions

When to follow new IA change process

For large data changes, not for small schema changes!

DB Change

Follow IA Change Process?

Delete a completely deprecated table

No

Removing completely deprecated fields

No

Adding small table for internal use not needed for reporting

No

Adding a few additional fields

Probably, use good judgement

Refactor: moving actively used columns to a different table

Yes

Adding complex tables that represent core business data

Yes

Refactor: renaming and replacing active used columns

Yes

What is a dot file?

  • DOT (graph description language)
    • Node, Edge and Graph Attributes
    • Basic of UML with DOT
    • Connections
      • Dir
      • Arrows
  • Easy to generate from Rails Model files
  • Easy to generate in multiple other frameworks
  • Easy to build by hand
  • Easy to “zoom” in on focus areas

What is a dot file?

Arrows (we need agreed on legend 1:1, 1:Many)

​

Dir

Proposed Legend

  • 1:1 -> “normal”

​

  • 1:Many -> “crow”

What is a dot file? Text Input -> Image Diagram Output

Example: Contact Cleanup

Refactoring data, attributes, & associations of Contact Models

Textual Diff

ZOOMED IN!

Refactoring out Phone Numbers

The Previous Iteration is Wrong

Issues:

  • DB currently has multiple users with the same primary number
    • shouldn't be allowed, but exists
  • DB intends to have many accounts with secondary numbers that are other's primary numbers.
    • this can & should happen for friends / family accounts
  • Previous design would have duplicate phone number records per unique number.
  • Therefor we couldn't appropriately link SMSs and Calls to PhoneNumbers without creating multiple records

Improvements

  • Contact -> ContactPhoneNumbers -> Phonenumber <- [SMSs, PhoneBlockList, Calls]
  • Remove Contact_Id and now PhoneNumbers is correct can map directly to tables with a number SMSs, Calls, & PhoneBlockList.
  • We can now see messages that went to individual users after a phone number changes as we can query between activate_at / deactivated_at.
  • We can also show when one message now goes to many contacts who all are sharing a number (hopefully as secondary numbers)

ZOOMED IN!

Workflow

1. Generate or update initial model on dev branch

* generate dot file: `FOCUS=true VERBOSE=true OUTPUT=doc/customers.dot rake diagram:models:customers`

* generate image: `neato -Tpng doc/customers.dot > doc/customers.png`

* check in the un-edited DOT & image files

* commit to dev branch

* this will serve as your before to diff against later

2. Create a feature branch to work on your changes

* new branch: `git checkout -b feature/db_phone_change`

* edit to reflect your changes: `open doc/customer.dot`

* convert updated DOT file to updated image: `neato -Tpng doc/customer.dot > doc/customers.png`

* view PNG & Refactor until happy

* create a PR and upload the before and after images, diff will be automatically created for the DOT file changes.

Workflow: Initial Model

  • If there is a suitable DOT file use it, if not modify `dev.rake` to create one

DB Refactoring