Evaluating Data Literacys: specifially the skill to manage and escalate critical information

Michelle Knight

Evaluating Data Literacys: specifially the skill to manage and escalate critical information

I am wondering if anyone has developed a means to evaluate Data Literacy in their employees? For example, is there a way to assess an incoming office assistant and their ability to escalate critical information, identify data quality issues, or ignore irrelevant information? Are there any thoughts?


Michelle Knight

[login to unmask email]

Freelance Production Assistant

Freelance Data, Technology and Science Writer

William McKnight

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Michelle Knight)

It's more for technical knowledge of subject matter, but reviewnet.net has tests you can put them through. 

As for application, I would pose various difficult scenarios and ask what they would do.

Frank Quintana

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Michelle Knight)

What is data literacy?

In general, the concept of literacy included reading and writing and perhaps counting (basic arithmetic).

Using this concept, data literacy could mean the ability to read data and to read data from a database means basic SQL queries or intermediate spreadsheet (Excel) proficiency probably both.

I think that sometimes data literacy is used as a synonym for data competence, in this case, the explanation is more complex since the level of competence must be different depending on the role and level;

Let me use mathematics as a metaphor to explain the "Data Competency". we all need arithmetic literacy (addition, subtraction, multiplication, and division perhaps also percentage). At the next level, maybe we need algebra and geometry and, depending on our profession, trigonometry, vector analysis, and finally we need calculus and perhaps everyone should now know the basics of logic, etc.

Unfortunately, there is no good definition of data competency.

An important question is at what level you should have your knowledge of Relational Databases, Datawarehouse, Data Integration (ETL, Data Pipelines), NoSQL, Data Modeling, Data Governance, Data Architecture, etc. In fact, some of these themes overlap.

If we want to establish an efficient educational system to achieve different levels of data competence; we need to define the different levels of competence and then define what the necessary skills at each level are.

Unfortunately, there is no standard curriculum for Data professionals at different levels and in different roles. Each college or university or professional associations, TDWI, IIBA, etc have their own.

Joe Danielewicz

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Frank Quintana)

What a good topic for discussion, especially in this era of AI and Machine Learning (ML).

Not long ago, only the data modelers, DBA’s and developers needed to know SQL. The Business Analysts needed Excel expertise and many of them did SQL which helped them develop the requirements for the next project. The data scientists certainly need to know the algorithms that go into ML but does everyone need that level of knowledge or do they trust the scientists to keep their biases out of the algorithms?

Since data is such a broad and nebulous subject, I humbly suggest that all the roles in IT should have a thorough grounding in the philosophy of data and information. I propose everyone in IT should at least read something about information theory, theories of meaning from philosophy, linguistics and semiotics. How do data models convey their meaning? What are the limits of expression of models and databases?

One of the best sources for these diverse topics is, Data and Reality, by William Kent (Kent 2012). Kent covers existence, identity, relationships and many other topics that are foundational to data literacy.

I’m currently working on a book that builds on Kent’s work and compares models to metaphors in how they express their meaning. I hope to use this forum to test some of my ideas and solicit critical feedback.

Thanks,

Joe Danielewicz

Ryan B

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Michelle Knight)

What a great topic. Especially since I made it one of my programs 2020 objectives.

I can share our approach, if it helps.

I'm using a tool we already had licensed called Oracle APEX and doing the development in house. I'm figuring most people have something they can use that's similar for a browser-based interface.

I've used it to produce 3 things. 

#1- About 20 questions I brainstormed with a few data stewards and some other stakeholders. The questions fall into categories like Data Quality, Security, Usage Rules, etc.., but they reflect our data world. Each question presents a scenario, a yes/no answer, and an explanation which might be a reminder of a policy or just a best practice (for us).

#2- A "Data Literacy Test" for business users. The questions go in random order, if you get 70% right you can print off a "Certificate of Data Literacy". The goal is to socialize the best practices and try to make the "test" easy and maybe even fun. Optimistic I'm sure, but I will expect every Data Steward to take and pass the test, and would encourage all staff to take it at their leisure. Might go around with a box of chocolate bars and reward everyone I see with their printed certificate.

#3- Tracking the results. I don't track the individuals, just the data on which questions were answered correctly. This is giving me two things. The first is insight into which questions or categories are strong or weak for the organization. That will help me focus on where we need to target our education & training. The second is how are we trending as an organization in our general literacy. That'll give me a metric on not just how we're improving but on the effectiveness of the DG program.

It's a work in progress, but I'm excited to look back in a year and have real data to work with.

Kathi Martin

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Ryan B)

I like the approach, Ryan. Are you willing to share your questions or your test? 

Ryan B

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Kathi Martin)

Do you want to just see some of the questions I ask? Or would you want more of an end-to-end demo? Some of the questions are only pertinent to our business processes and systems, but I'm happy to do either if you're interested.

Laura Kostyo

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Joe Danielewicz)

Glad to see you on this forum and know you are still active.

Kathi Martin

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Ryan B)

If you're willing to share the questions as a start, that would be great. I understand they are specific to your organization but might be a start. Then later maybe I would take you up on your offer of a demo. Thanks!

Kathi D. Martin
Senior Director, Advancement Records
The Ohio State University
614-292-7256
[login to unmask email] osu.edu

Tolu Oyebode

RE: Evaluating Data Literacys: specifially the skill to manage and escalate critical information
(in response to Ryan B)

Hello Ryan,

Similar to Kathi, I am very interested in the questions you developed for your stewards and business users and your tracking mechanism. I am also involved in rolling out a data literacy campaign in my organization and looking for resources to use. I would love to benchmark some of your ideas if you don't mind. 

Thank you.

[login to unmask email]