Quran text verification framework

husayt

While working on the Quran Meta Library, we've encountered a variety of small issues over time in our calculations, particularly with unexpected inputs. To address this, in the latest version, we prioritized achieving 100% unit test coverage to eliminate any chance of error. This rigorous approach allowed us to uncover and resolve numerous hidden issues, significantly improving reliability across Koran.center.

My next focus is on the accuracy of the Quranic text itself. We source texts from multiple providers, including AlQuran.Cloud, and we’re looking to implement a robust verification framework to ensure absolute accuracy. Given the variations (e.g., Imla’i and Uthmani script styles, with or without specific diacritics), having a universal verification system that supports multiple encodings and font styles is essential. This would help guarantee that the text appears correctly across all platforms.

Additionally, the challenges related to font compatibility are unique and complex, so we are planning a separate project to address these issues. It would be fantastic to collaborate on these initiatives, not only to enhance our individual projects but to protect the integrity of the Holy text on the internet from any unintended or malicious errors.

meezaan

husayt Thank you for sharing the information and the idea.

Collaboration is always a positive thing, Alhamdulillah.

Any verification work on the Qur'an to ensure accuracy would require 2 components, in my humble opinion:

Scholarly; and
Technology to actually implement the scholarly work.

The second, whilst not always easy, is almost always do-able. For the first, we need a larger group to collaborate with, unless you already have people involved with Koran Center?

Thanks for sharing again!

husayt

Absolutely, both directions are important. The problem is that scholarly verification while is so important, very few websites will do it. So we have most of websites getting their texts from each-other and relying on others to do it. What we want to achieve is some kind of chain of verification, which will guarantee, that once we have a text which has been scholarly verified, and it has been then copied and used in other places, that no corruption occurred in the process. Some kind of tooling which we can use against golden scholarly verified source. Also we can then even cross check verified copies as well. So we will have a repository of both text and tooling. Also it is essential to provide a metadata against each shared text of Quran, in regards to where it is from, who it has been verified by. To make sure that no unverified texts of Quran are used anywhere.

For instance, we can start by adding this metadata to texts of Quran shared via Quran.cloud.

From Koran.center we can share the transformation framework we use internally to make sure that each font is properly rendered on the screen.

husayt

Here is similar question https://community.islamic.network/d/80-regarding-the-review-and-verification-of-quran-text-on-httpsalqurancloud on this forum, which proves my point. We say the source is tanzil, same time i just did comparison between text on tanzil and api and I can't see they fully match, it seems some kind of transformation already happened, or perhaps tanzil has updated text since it was imported here last time.

This is a complex task and indeed requires whole islamic institutions to take charge, but in their absence tech community can provide some tools to at least do the first level of defence.

meezaan

husayt The source is Tanzil, but there were typos identified in the text which were subsequently fixed. Tanzil does not really respond to queries, so it's difficult to know if they even update the texts.

The task is indeed complex, but it begins with an authority saying this is correct. But if other Qurans in common print show something different, that credibility can quickly come into question.

I understand the view about line of defense, but we need to have a common denominator to defend.

For instance, we can start by adding this metadata to texts of Quran shared via Quran.cloud.

This is a good idea, I should think, as it can only enrich the information.

husayt

in future we even can even share quranic text in some blockchain like format which is properly versioned, frozen from changes, distributed and integrity guaranteed. Git/github is actually good too, since it has timeline of changes. The problem with api shared text is that even verified text can change over time, due to some corruption in database or api layer.

meezaan

husayt I think blockchain is definitely something to steer clear of. A decentralized version of of Quranic tex authority - that's a recipe for unpleasantness, I think. Integrity guaranteed by whom? And how do you push fixes easily?

There are other simpler ways to guarantee integrity, using checksums, for instance. APIs are also okay, provided there is a good backup strategy. But https://github.com/dolthub/dolt this is actually a very good answer. It just needs to be tried with Doctrine, and if it works, it's a drop-in MySQL replacement.