Baselines

Everything scored by the benchmark implements one contract (pic2smiles.evaluation.harness.Recognizer): given image paths, return one prediction each. A system that cannot run returns skipped with a reason and is excluded from every rate rather than scoring zero.

Reference points, not competitors

name	what it is	why it is in the table
oracle	reads the answer from the manifest	validates the harness. If it is not 1.000, the scoring code is broken, not the model.
image_hash_nn	average-hash nearest neighbour over a rendered gallery	the memorization floor. It can only return a molecule it has already seen, so it shows what "retrieval without recognition" scores on each tier.

External OCSR systems

name	approach	status
MolScribe	Swin encoder + graph decoder	runs on Linux via subprocess; cannot run on this macOS 27 laptop (see below)
MolNexTR	ConvNeXt + transformer, graph output	not installed
MolGrapher	atom/bond keypoints + GNN	not installed
DECIMER	transformer image-to-SMILES	not installed
Img2Mol	CNN encoder + pretrained latent decoder	not installed
OSRA	rule-based vectorization	binary not installed

MolScribe on macOS 27

The Python 3.10 environment that previously ran MolScribe no longer imports:

ImportError: dlopen(.../scipy/sparse/linalg/_propack/_spropack.cpython-310-darwin.so):
  section '__DATA/__thread_bss' has a zero-fill section type, but offset field is not zero

macOS 27's dyld rejects the older compiled scipy extensions. The fix would be scipy >= 1.16, which requires Python 3.11+, while MolScribe pins torch<2. Upgrading scipy inside the 3.10 environment does not resolve it.

This is reported as a result, not worked around: part of the goal is a model that runs on this laptop, and the strongest available pretrained baseline currently does not.

Subprocess boundary

MolScribe's dependency stack (old torch, albumentations, scikit-image) cannot share an environment with a current PyTorch training setup. The harness shells out to scripts/molscribe_predict.py under a separate interpreter, which prints one SMILES per input image. A missing interpreter or checkpoint degrades to skipped with the path recorded.

Numbers from the previous revision of this repo

The earlier report listed MolScribe at 0.000 exact match on 20 hand-drawn images. Those numbers came from the environment that no longer runs, on a sample of 20, and are not carried forward. Any MolScribe figure in the current report was produced by the current harness on the current frozen evaluation sets.