Leveraging Machine Learning to streamline document processing

Leading occupational healthcare services provider Jobfit was - in partnership with Clade, able to develop a redaction solution using Microsoft’s new Azure Cognitive Services Vision API for Optical Character Recognition (OCR), to enable document review and data entry, while protecting patient personal information.

Jobfit, part of the Fullerton Health Group, is a leading occupational healthcare provider, delivering a wide range of services, including pre-employment medical assessments, work fitness assessments, drug and alcohol screening and onsite medical services.  Currently, where clients have medical checks performed via external non Jobfit providers there is a time-consuming task to perform electronic data entry of medical information from those providers.  To streamline this process and ensure medicals and employment suitability checks are not delayed, Jobfit have been investigating the use of third-party providers of data entry services.  This option has not been possible in the past, due to Jobfit’s diligence in care of their client’s medical information. No personally identifiable medical information can be shared with any party without consent of the individual, as mandated in the Privacy Act, and manual removal of personal identifier’s from those documents is too time consuming to allow Jobfit staff to respond to the changing volume of documents while maintaining medical check delivery targets.

Partnering with Clade Solutions, they were able to develop a redaction solution using Microsoft’s new Azure Cognitive Services Vision API for Optical Character Recognition (OCR), this solution enables document review and data entry, while protecting patient personal information.  In summary, this solution uses Machine Learning services to identify and remove individual identifiers (e.g. Name, Dates of Birth, Address, etc) within scanned documents.  We explore this in further detail as we sit through an interview with Trevor, one of Clade Solutions’ senior consultants.


Trevor, what exactly is OCR and the automated redaction service? How does it work?

The use of OCR and the creation of this service was done to allow automated redaction within pdf documents. When a pdf document is uploaded the system will extract all pages and send those to the Microsoft Vision API to extract text and position information. We then have a recognition engine, which will ingest the Vision API result and identify specific areas which need to be redacted based on either labels that surrounding that information (in printed forms) or personal detail text itself. Be it a person’s name or key phrase, it would redact that area and then generate a new non-identifiable document.


What was the issue that the customer had approached Clade with?

Physical documents need to be data-entered into a digital platform in a time-sensitive fashion.  In order to deal with fluctuations in the number of documents to be processed (scaling up/down data entry operators) external data entry specialists could be used, however the personal nature of the information to be data-entered had precluded their use.  A solution was required within a short time frame (due to increasing numbers of documents to be processed) which can guarantee that personal patient information is protected.

There are some existing OCR/redaction solutions which available from third parties which Jobfit also reviewed, however they were cost prohibitive and did not meet all requirements. Jobfit needed a cost effective and tailored solution, in a timely fashion, and therefore asked Clade to assist them in creating their own.


What do you believe have or will be the three biggest benefits of the OCR and redaction service?

The biggest benefit is that Jobfit remains compliant with their responsibility to protect patient data, but is able to scale their ability to data enter that information without delaying medical check results. Another benefit is cost, we were able to provide a solution at 20% of the cost quoted by other vendors. Finally, our automated solution is efficient and less error prone than manually redaction, while freeing up staff for more important customer service tasks.  So we were able to save company resources in terms of time and efficiency, while enabling Jobfit to focus on customer service rather than administration.


What technology was used in the process of creating this solution?

In the case of this specific solution, we’re leveraging the Microsoft Vision API, an online service provided as a part of the Microsoft Azure platform. Vision is performing the OCR side of the solution. Essentially, we’re giving it an image and then its finding all the printed text within that image. We have a custom solution responsible for interacting with that API; extracting images from pdf documents, managing a Vision API queue, and taking the OCR results and identifying regions to be redacted based on text and region. We have also custom developed a custom review engine web application for staff to review redaction region results. Once the redactor has identified all regions it believes need to be redacted, the web-based tool allows staff to review those locations, modifying and removing any redactions if needed. The engine itself is flexible in terms of finding common locations where the redacted information needs to be; allowing us to work out what type of document is being scanned based page/document content. We then define the rules to hide information depending on where it is. Once a staff member has confirmed the document to be non-identifiable the last stage of the process is to apply the redactions to the original page images and assemble a new document with those areas located and redacted out.


With all of these advanced processes going on in the background, is using the application itself simple?

It’s a simple to use application; just provide it with a document, where the system automatically goes into background processes and pulls out locations it assumes needs to be redacted. After this, it’ll automatically generate tasks to review each document, where all employees need to do is click on the review button to view the document with all redacted locations. They can then just click a button to confirm that they’ve checked the entire document and it’s done. However, with the inputs within the engine varying, there could be tasks in the future to go through which may require us to tweak the engine for the type of document if it consistently misses redacting pieces of information. But so far in the two months it’s been running, there haven’t been any requests to make changes just because the generic case is working so well.


Any final words or future predictions about what this type of technology offer?

Well, the underlying technology is all based on machine learning and AI, these are the kind of solutions that couldn’t really have been done successfully even a few years ago. So certainly, it is opening possibilities for new enhancements in terms of efficiency of workplace that couldn’t have been implemented before.  The underlying Microsoft Vision API OCR as a service certainly makes this kind of solution much more efficient to implement; instead of having to create a machine learning solution to perform the underlying text recognition, we could focus our efforts on the business required identification of key text instead.

We are immensely thankful to Trevor for giving his time to answer these questions. From success to success, Clade Solutions continues to aim high in providing real-time expert solutions.