Q: How can I evaluate qualitative data from different sources consistently and stringently without becoming too subjective and making too many assumptions?

Here are a few ways in which you can proceed. Note that these are relevant for evaluating quali data in general. We haven’t considered Industry 4.0 cases in particular.

• Assess the background and qualifications of the person providing the qualitative data. Does the person have the necessary knowledge, qualifications, or experience to make those judgments? Or has the person had that experience long enough to provide reliable data?
• Check whether the same data have been reported by several persons who are unlikely to be connected to each other. If various unconnected people are reporting the same data, it is likely to carry greater credence.
• Interview people providing the data – to form an independent judgment about the veracity of the data.
• Check for patterns – are the same types of data being reported from a specific geography/demographic group, etc. – and analyze factors that may be responsible for these patterns. By doing so, you can isolate data that are outliers to these patterns and further probe/verify inconsistent data.
• Try to create test situations using the qualitative data, and verify whether the results of these tests are predictable/as expected.
• Check if there is any precedent that supports the data.
• If possible, hire an independent agency to collect data and check if it corroborates the data you already have.

Hope that helps. And all the best for your project!

In addition, you may find it helpful to go through these related queries by other researchers:

• What are the possible problems that may be encountered in Qualitative Research?
• How should I present qualitative interview data in a research article?
• How to write the Discussion section in a qualitative paper?