You have a large set of data. Several segments of a large audience have overlap in which data may be useful to them – some of which they don’t have access to currently and therefore no model for usage, and some of which they may think about differently from other segments. You want to enhance their access to, understanding of, and increase their personal productivity with these new and existing data.

Create a conceptual mapping of your best understanding of the following with all the resources you have: Audiences A, B, C, etc. – motivations, needs, behaviors.

Then connect these to the data pieces they consume, what they do with them, and what they help them achieve.

Your choices in defining solutions are to:
1. Improve access, usage tools, and interpretation of currently accessible data.
2. Augment currently used data with new data to create new meaning and actionable intelligence.

As you create the above mapping, possible solutions or features should start making themselves evident as they would help you to connect the dots better. The tough part is prioritizing, and you may need some stakeholder/user input to help with that. Also crazy check with a colleague as frequently as possible. Eventually you’ll be able to circle a prioritized audience; a set of needs, motivations, and behaviors/tasks; and a features/solutions set. You should have a secondary set of all of the above as well, and you’ll want to keep these in mind as you implement the solutions for the higher priority items.

It might be interesting to separate the status quo from the end goals of each user type as they interact with the data set. What are they using it for?

From there, I would focus on determining whether the data set can even support those end needs well, as it exists today and if not, determine the resources it’d take to get the data in the shape you’d need it and whether those resources can be secured. If that can’t be done, you might be stuck with planning a band-aid rather than an ideal solution — but you’d want to know that before you got too far along.

And then, if the data set is adequate, you can identify a high-level long term plan and work with the team to prioritize the first chunks that should be tackled. The organization should help dictate the phases based on their ROI.

That might sound idealistic and I know I don’t understand your situation completely, but I hope it helps.

I’ve been in rooms where people willingly argue for hours in the abstract without anyone making a sketch on the whiteboard. Why? They don’t want to expose their intuition and they’d prefer to argue endlessly because it’s on logical turf. Then I made a sketch of a UI and ask “did you mean this or this?” and suddenly productivity and clarity happens. Design is visual. Sometimes once you see it you know with certainty those huge concerns you had aren’t really that scary anymore.

So my advice is draw something. Sketch something. Pick a small enough part of this problem, however small it is, and let your intuition loose. Then at least you’ll be arguing about solutions, instead of obsessing about the right framework or model for the problem, which you likely won’t sort out until you’ve done a bunch of sketches to help sort out your thinking.

If overwhelmed. pick the common cases. Or even one common case. But design. And sketch. And try. And work it until it’s something that is limping along.

Once you have something, however small that doesn’t suck, add another constraint or scenario in and see what happens.

Then go back and read your blog post again. I bet you a $200 dinner you’ll romp your way through the questions you listed in your post.

This leads me to think that while the design of this sucker is a big honkin monster, you’re primary challenge is actually one of moving people within an organization and you should plan accordingly. I suggest you pick a quite discreet subset of Audience X and develop a quite discreet subset of functionality to build initially. You should be totally pragmatic and completely selfish when selecting your first set of users: it should be whoever is both most likely to understand the value of your work and will do the best job of yakking about it to the rest of the organization.

If in your initial launch you make public your definition of the initial users as well as how you are segmenting all the other users, you might be able to build up some healthy interest/competition. And you may be able to convert that interest/competition into guidance for who you roll out to next. Since your audience has never had anything that does what you’re trying, iterative design cycles should work throughout the project. The lack of previous solutions frees you of the burden of delivering complete functionality. Aim for small and incomplete.

If you take my suggestion, you’re basically following the model for small businesses in a capitalist economy. You launch a simple product line and add to the complexity of the product to increase your customer base while also launching new products as demand surfaces.

You’ll have to manage through any resentment issues between user sets, but remember that dissatisfied potential users are great subjects for user research on the fly, converting their frustration into your prioritized requirements for smiles all around. If you get serious heat from any of the groups demanding greater functionality, you can tell them they can have anything they want as soon as they pony up the resources to feed development.

Does that help?