Agile data modelling/modelling in an Agile project

Bridget Keam

Agile data modelling/modelling in an Agile project
Is there any good way to approach this? What are some of the challenges? I've always been of the opinion that data modelling (at least conceptual) should be done during discovery. Interested in experiences.

Anthony J Algmin

RE: Agile data modelling/modelling in an Agile project
(in response to Bridget Keam)

I would classify myself as a Monday-morning modeler (not a modeling specialist), but most of my experience in this space has been more Agile-oriented. Whenever I or my teams have been tasked to create data solutions, I push us towards a rapid-prototyping MVP state--which will necessitate a bare-bones model that will need to be grown and nurtured over time.

The art is in creating extensible early-models that easily accommodate future revisions and enhancements. The closer the specific decisions are to the end data consumer, the more effort should be made to get it "right" before exposing it. It's much easier to adapt upstream processes to an evolving model than to ask people to rewrite their connections to the data. I use an API-like approach, where once it is there and available I really want to keep it there and supported unless the reason not to is significant.

Anthony J. Algmin
[login to unmask email]
Go Make an Impact!

William McKnight

RE: Agile data modelling/modelling in an Agile project
(in response to Anthony J Algmin)

I really like how Anthony put it!

Remember it's still conceptual-logical-physical. I suggest doing the conceptual asap for the enterprise (i.e., your forseeable scope of the enterprise) and filling in through physical in pieces in an agile way, modeling only those pieces you are going to populate with data - and use  - in production - in the next month or so.

Aaron Fuller

RE: Agile data modelling/modelling in an Agile project
(in response to William McKnight)

William, I agree that it's very useful to get a conceptual enterprise model figured out ASAP, but sometimes ASAP is slower than the Agile dev team needs to move. I'm having good luck creating partial conceptual models that support just a little bit more than the logical/physical models we're creating for the agile iterations. Then I work ahead one sprint on conceptual modeling, basically. Sometimes you just can't spare the time immediately to build out the conceptual model completely. If that means a little bit of refactoring later, so be it.

Aaron Fuller, CBIP

Founder & Principal Consultant

Superior Data Strategies 

517-803-0714

superiordatastrategies.com

<a href="https://www.superiordatastra

Walter Howard

RE: Agile data modelling/modelling in an Agile project
(in response to Bridget Keam)

The challenge is you only get to see a very small subset of the data requirements.  Each sprint adds on to the previous sprint until you hit the "moment".  And the moment is when everything you've built up to that point is actually incorrect based on the new information in the latest sprint.  You promptly go to the scrum master and inform him that the current model needs extensive rework to bring it in alignment with the latest requirements.  "We have no time for that" is the only answer I've ever received.  The end result is a half baked database solution that can be more confusing than informative.  Perhaps I need to work on my persuasion skills.

Lawrence Witner

RE: Agile data modelling/modelling in an Agile project
(in response to Bridget Keam)

Hi Bridget,

I am a relatively new data modeler, 2.5 years.  I work for an organization, however, that is trying to become agile so this is something I think about quite frequently.  In my organization it has been a struggle to get the data modeler involved at the beginning of a project.  We rely heavily on the developers.  Using scrum meetings, however, I am able to collaborate with developers more easily and get involved at the beginning of projects.  I love Walter's post, and I agree with him.  Nobody likes iterations.  Lol.  But they also don't like finding out they have to change something at the end rather than at the beginning.  Part and parcel with iterations is the idea that your work isn't perfect and it has to be done over.  I am finding that with agile we have to learn to be comfortable being uncomfortable.  I think of agile as having four primary characteristics (collaborative, iterative, immediate, and retrospective).  I find that collaboration is the most valuable and that it helps smooth over immediate and iterative values.

Aaron Fuller

RE: Agile data modelling/modelling in an Agile project
(in response to Walter Howard)

I think that if the scrum master says "we don't have time for modeling" then they haven't fully bought into agile. Being agile means doing requirements, design, development and testing just-in-time. It doesn't mean doing requirements and design all up-front. We have to make time throughout the entire project for modeling, that's all there is to it.

Aaron Fuller, CBIP

Founder & Principal Consultant

Superior Data Strategies 

517-803-0714

superiordatastrategies.com

<a href="https://www.superiordatastra

Peter Cohen

RE: Agile data modelling/modelling in an Agile project
(in response to Aaron Fuller)

Aaron - I couldn't agree more.  I was the lead data architect on an agile data warehouse project for a Federal Government agency for 2 years.  When I started they had a very similar attitude but I worked a way to provide an "agile" approach to data modeling for every sprint while maintaining a holistic view of the entire 600 table data warehouse.  A good data modeling tool such as Erwin or ER/Studio will allow this.  I supported several Scrum teams using my methodology.

Lawrence Witner

RE: Agile data modelling/modelling in an Agile project
(in response to Bridget Keam)

I recently read an article that really made me think.  It was a B2T article on "Agile Project Metrics Worth Capturing."  One of the takeaways from the article is that there should be an improvement in employee satisfaction with Agile.  I think this is because of its emphasis on collaboration.