When and how should resource validation happen in a FHIR-based information exchange?
Those familiar with the FHIR specification will know that FHIR provides several ways to validate whether a resource conforms to a FHIR Structure Definition, whether that StructureDefinition is a profile or a base resource definition. If the resource is in XML, the syntax can be validated against FHIR’s XSD. FHIR also provides a more in-depth procedural validator that checks additional factors, such as invariants. The procedural validator has its own limitations, but for the purpose of argument, let’s assume the validator can determine if a resource conforms to a specific StructureDefinition.
In terms of how validation happens within FHIR workflows, we need to distinguish two use cases. When a client requests a resource of a particular type, or requests resources conforming to a particular profile using profile search, that client may use validation to be sure returned resource is indeed what it asked for. This is the solicited case. If a resource is being sent to a server as a create or update, the server may want to assure that the resource conforms to one of the profiles supported by the server, as declared in its conformance statement. This is the unsolicited case. The difference is, in the solicited case, the client knows what StructureDefinition the resource ought to comply to. In the unsolicited case, the server does not have that information.
The solicited case is, on the surface, rather straightforward. When the client receives the requested the resource, it can call the appropriate validator and determine whether the returned resource conforms to the expected StructureDefinition. It the client did a profile search, it knows exactly which profile to check the resource against.
The use of includes and reverse includes in searches, however, complicates the issue. One cannot specify a profile those searches, only the base resource type, and therefore, all that is known about the resources returned as part of the include is the base resource type. The client does not know that the included resources conform to any profile the client supports, and furthermore, the client does not necessarily know which profile(s) to check against. This is the same situation as in the unsolicited case.
The unsolicited case is more complicated. The server may want to validate that the resource conforms to one of its supported profiles, as declared in its conformance statement. The problem is, which profile should the server use to validate the profile?
Sometimes the answer is clear. If the server only supports one profile on Medication, for example, and it has received a Medication resource, then clearly it knows to validate the resource, since there is only one Medication profile. It is not always so simple. On certain resources like Condition or Observation, there may be many profiles. Then, the choice of which profile to validate against is not so clear.
The brain-dead approach would be for the server to iterate through its library of profiles until it finds one that validates the instance. This approach assures that, in the event of a non-conforming resource, every profile based on the given resource type will be tested before the incoming resource can be rejected. If there are 100 different Observation profiles, that might take a long, long time. Ugly!
An alternative would be for the receiving system to “pop open the hood” on the resource, dig out the primary code, and try to determine which profile to validate against, based on that code. An observation might have a LOINC code, for example, that indicates it is a hematocrit score, and that may guide the server to pick the hematocrit profile to validate that resource. While this might work, it (1) requires processing the body of the resource before it is validated, which is exactly what validation tries to avoid, and (2) requires the server has to have a mapping that connects codes and profiles, which means extra work and maintenance.
The third (and best) alternative in the unsolicited case is for every incoming resource to declare which profile should be used to validate it. There is a metadata attribute, Resource.meta.profile, where the profile declaration can be placed. Then, when receiving the resource, the receiving systems only needs to look at the declared meta.profiles, and test against those. At least one element of meta.profile must name a profile that the receiver supports, otherwise the resource can be immediately rejected, without further ado.
But… there is a catch. In FHIR DSTU 2, resources are not required to fill in the meta.profile slot. In my opinion, for all the reasons discussed above, the information sender should always declare which profile(s) the payload should be validated against, naming one or more of the profiles supported by the information receiver. This simple expedient allows the receiver to immediately address validation, without iterating through multiple profiles, or guessing which profile(s) to test. This requirement could be stated in an implementation guide, or it could be done with profiles, by changing the cardinality of meta.resource from [0..] to [1..].
Should it become standard practice in the FHIR community for every resource to explicitly declare which profile it should be validated against?
Light My FHIR 
No. it shouldn’t. This is another example ‘because I want to do this in my context, everyone should be forced to do it their context’.
You described 3 different ways that this could be made to work. So why force all implementers to populate the profile id when there’s other ways? If you want to make people fill out the profile id, then you can declare that in your base server conformance profile. But forcing everyone to do that all the time – that’s not appropriate.
And, in fact, often implementers are not conforming to a single profile, or any profile at all. Or doing validation using the formal validation. We have the tools to allow people to specify what behavior they want.
LikeLiked by 1 person
Grahame, don’t misunderstand me. I am not proposing a change to the standard. I agree that people can use FHIR for many purposes and in ways that will not require validation. Those who choose to use validation can do so unilaterally, by mutual agreement with a trading partner, or by creating an implementation guide incorporating validation requirements. So, I should have prefaced everything above with “If validation is desired…”.
But the gist of the post is still the same: to make validation work, the most feasible approach is for each resource to identify the profile it should be validated against. The key use case is when a server that supports many profiles of a certain resource receives a create or update of that resource. In that case, how does the server determine that that resource conformed to a supported profile, if the server wishes to validate? The trial and error approach doesn’t scale, and interpreting the contents of the resource before it is validated, to try and figure out the right profile to validate it against, seems backwards. That leaves the method suggested, i.e., that the sender indicates the profile the resource claims conform to, in meta.profile. Do you have another approach to making validation work? If you have another idea, please do share.
LikeLike
well, it definitely has it’s place as one way to drive validation. But it has it’s limits too – the server trusts the client to pick the right validation for whatever purpose. What if it doesn’t pick the right thing? And, in fact, so what if it does? when the server actually goes to use it for something, then it knows what to check it against. If the server never uses it, why does the server validate it at all?
You shouldn’t dismiss the other architecture – code driven profile validation – so lightly, either. In this approach, the server validates the base resource first. Then it does a use case based validation because of some discriminator property in the profile. We’ve not paid sufficient attention enabling this, but I think you didn’t consider it enough.
Finally, I think many servers will work in the background identifying multiple profiles that a resource conforms to, not just one. There’s all sorts of reasons to take that approach
None of this is to say that what you propose isn’t the right approach for a particular deployment, and I’ve seen deployments that work just as you propose. They do seem to create closed trading schemes, where other general content is excluded, because it’s not in on the agreements concerning particular profiles. So an alternative is to allow the client to mark the profile to save the server time, but not to insist on it. but maybe such a closed trading system is what you want…. then you’ve defined how to make it work efficiently.
LikeLike