How do I break up my data models into smaller chunks?

Michelle Knight

How do I break up my data models into smaller chunks?

How do I break up my data models into smaller chunks? This question came up at the Data Modeling Best Practices - Business and Technical approaches webinar today.

From what I heard, Donna Burbank cautioned about getting into too much detail with Data Modeling. She advised against having Death by Data Modeling (The three walls with complicated diagrams that take a couple of all-day meetings.) To be more effective, Donna suggested breaking these Data Models in chunks.

Responses to these questions mentioned breaking the business into subject areas, following the use case from the Agile Sprint, and focus on some core concepts. Does anyone else have thoughts or tips and tricks?

Presentation slides will be available by end of day Monday, 10/28/19.

Edited By:
Michelle Knight[All DATAVERSITY Members] @ Oct 24, 2019 - 03:42 PM (America/Pacific)
Michelle Knight[All DATAVERSITY Members] @ Oct 24, 2019 - 03:47 PM (America/Pacific)

Thomas Fontanella

RE: How do I break up my data models into smaller chunks?
(in response to Michelle Knight)

I am afraid that I don't have solution ... but I completely agree with the problem! In our case we are trying to make data models available for our commercial software systems. But no one needs to see the whole model - and we'd need to rent a billboard to display it anyways. The vendors have not been able to provide those subject area chunks either. It seems like an obvious need.

Merrill Albert

RE: How do I break up my data models into smaller chunks?
(in response to Michelle Knight)

I don't think huge wall-sized diagrams benefit anyone.  It's too much to absorb, too convoluted to follow lines, and too time-consuming to manage.  Subject areas are so much easier.  Each subject area is a topic of conversation.  You have not only the entities for that specific area, but all entities it touches in other subject areas.  That gives you the link to other diagrams to go to for understanding other topics.

Michelle Knight

RE: How do I break up my data models into smaller chunks?
(in response to Merrill Albert)

Thank you, Thomas and Merrill. Upon doing some research and writing the concept of knowledge graphs or property graphs seemed viable. I picked up some diagrams from Thomas Frisendal. From my software testing and information architecture experiences, I used Microsoft Visio to do this sort of mapping. I remember it clarified which concepts or operations could be chunked to make running a test or putting up a website more effective. Has anyone had hands-on experience with this sort of graphing? Would a property graph be a good tool to see data model per subject areas, Merrill?

See the attached image.

Michelle Knight

[login to unmask email]

Freelance Production Assistant

Freelance Data, Technology and Science Writer

Attachments

  • property_graph_example.png (111.1k)
  • property-graph-definition.png (150.6k)
Edited By:
Michelle Knight[All DATAVERSITY Members] @ Nov 26, 2019 - 09:57 AM (America/Pacific)

Merrill Albert

RE: How do I break up my data models into smaller chunks?
(in response to Michelle Knight)

My preference is always logical relational models.

Walter Howard

RE: How do I break up my data models into smaller chunks?
(in response to Michelle Knight)

For agile development, you build the data model to support the current sprint, no more, no less.  The size is based on the number of stories in the sprint.

Once the application is complete, the model may be quite large.

And Donna is correct, no one likes to sit through large complex data model reviews.

The solution I've adopted over the years is to take a process based review of the model.  This is akin to the same methodology that was used to build the data model.  Common processes such as Create new Customer, Take an Order, Create an Account, all make sense to the business community.  Grouping the underlying entities, attributes, and relationships by process should comprise a reasonable, comprehensible "chunk". 

Phil Forestall

RE: How do I break up my data models into smaller chunks?
(in response to Michelle Knight)

Lots of good answers on this post. I suggest you start by breaking up the data modeling over time. Design from the general to the particular, i.e. first conceptual, then logical, then physical.

As much as conceptual data modeling can contribute to solution architecture, like all conceptual modeling it should also inform project planning and budgeting. A conceptual data model helps improve project planning early by :

  • defining data scope
  • defining common data vocabulary
  • sketching data complexity, volumes, sourcing and integration
  • revealing project demands for expertise and time allocation
  • generally helping to understand project issues and risks

Suggestions for conceptual data modeling:

  • buy, copy (legally!) or build a common conceptual data model for all development in your organization:
    • entity names to be business-oriented, definitions from respected outside sources; model includes credible references to external literature
    • entities-only (no attribution!) because when the focus turns to fields, nobody can talk about anything else, you're doing logical modeling, and the time requirement just blooms
  • divide the data space into subject areas:
    • subject areas are internally cohesive; they should be relatively self-contained
    • not too tightly bound to the other subject areas
    • maintain the subject areas in subsequent logical modeling
  • every development project links to this conceptual model as-is:
    • so the entities in play in the project are pre-defined
    • new entities from the project can be adopted, with thanks, in the conceptual model
    • changed entities are scrutinized very carefully; why is this project different? what are we learning here? this is important governance
    • so conceptual modeling takes days (if not hours) and cheap enough to do regardless of expected project size

There's no reason not to do a conceptual model for OTS product acquisition as well as new development.

Best wishes.

Thomas Fontanella

RE: How do I break up my data models into smaller chunks?
(in response to Phil Forestall)

Great advice from all - thank you! We have informal agreement on subject areas so it wouldn't be starting from a blank slate to associate entities to them. We are also in the process of selecting/deploying a data catalog w/business glossary which may help to document and communicate the model.