Potential identity calculation bias related to window position (v0.9.8)

[justdx]

Hello,

I observed unexpected identity values in ModDotPlot (v0.9.8) that appear to depend on window position. To test, I designed a controlled simulation.

Test Setup: 3 windows (w1, w2 and w3, 10 kb each)

Pairwise Identity in the initial run:

• w1 vs w2 → 96.8%
• w1 vs w3 → 85.0%

Simulated Sequence (300 windows total)

w1 × 1 copy + w3 × 100 copies + w1 × 99 copies + w2 × 100 copies

Parameters: -w 10000 -id 0

Observations

1. Inflated Identity

Expected: 85% (w1 vs w3) and 96.8% (w1 vs w2)

Observed:

• Window 1 (w1) vs Window 2 (adjacent w3) → 96.8%
• Window 1 (w1) vs Windows 3–101 (w3 copies) → 96.5%
• Window 1 (w1) vs Window 201 (first w2) → 96.8%
• Window 1 (w1) vs Windows 202–300 (remaining w2 copies) → 83.8%

2. Unexpected Zero Identity

Expected: 85%

Observed:

• Window 2 (w3) vs Windows 103–199 (w1 copies) → 0%

3. Filtering Behavior

When using: -id 70

ModDotPlot removes:

• Rows with identity = 0 (expected)
• Rows comparing Window 1 (w1) with Windows 3–101 (w3 copies)

The results suggest that identity values may vary depending on window position or ordering, producing inconsistent identity estimates for identical sequence comparisons.

I can provide the simulated dataset and command details if needed.

Thank you for developing ModDotPlot.

Xiao

[alexsweeten]

Hi Xiao,

Thank you for bringing this to my attention and for the detailed experiment setup! I think the results you’re seeing here make sense.

ModDotPlot computes sequence identity by using overlapping windows. Given a window size w, it computes sequence identity using a window of size 2w, (expanding by w/2 in each direction). We explain this further in our paper. The idea behind this is that overlapping windows improves identity estimation in tandem repeats larger than the window size, such as the 45s rDNA arrays in primates.

Going back to your experiment, the pairwise identity between w1 and w2 should be higher than between w1 and w202. This is because w2 neighbors w1, so half of w1 is used in the identity estimation. The pairwise identity between w1 and w202 for will be lower, since no duplicated regions of w1 are used in the identity estimation.

If you do not want this behavior, you can control the amount of overlapping sequence between windows using the --delta command line argument. The default value is 0.5, meaning half of each neighboring window is used as overlap (w/2). If you run the same experiment with --delta 0, then the identity between (w1 - w2) and (w1 - w202) should be identical.

Hopefully this explanation makes sense!

• Alex

[justdx]

Hi Alex,

Thank you for the clear explanation, that makes sense. I should have read your paper and not naivly assumed there's no overlap just because the BEDPE regions don't overalpped.

For my centromere analysis, repeat units are <200 bp and I use a 10 kb window size. In this case, allowing overlap can increase identity between adjacent windows. Setting --delta 0 appears more appropriate for my analysis.

I re-ran the same simulated dataset with: -d 0

The results are now much more consistent with the expected identities, so this resolves the main issue.

However, I have one small remaining question.

With --delta 0:

• Window 2 vs itself → 100%
• Window 2 vs Windows 3–100 (identical w3 copies) → 99.992417...%

Since these windows contain identical sequences and overlap is disabled, I would expect the identity to be exactly 100%.

Could you clarify what might cause this small deviation?

Thank you again for your helpful explanation.

Best,
Xiao