How to resolve misplaced text and other visual aspects of a PDF manipulated using redactions and insert_htmlbox.

[mgalvank]

I am trying to use pymupdf to help translate a foreign language pdf document to an english language pdf document but also trying to maintaining the formatting the best I can. This involves maintaining the text location, font color, font styling (bold, italics), annotations (strikethrough, underline), hyperlinks, images, tables, etc.

I am working with this example Chinese document -
CHINESE.pdf

Here are the output translations -

1. Option 1 -
   CHINESE_OPTION_1_custom_redactions_with_dict.pdf

2. Option 2 -
   CHINESE_OPTION_2_translated_custom_redactions_with_blocks.pdf

Issues -

1. For some reason the text ff is missing throughout the pdf. Unsure why this is happening. This text is searchable. Searching for Affairs highlights the text. Here's the screenshot -
   

2. Final pdf is not consistent through Option 1 and Option 2. I like the Option 2 output for the Chinese document in question but this same logic fails when the text is spaced out in the line say in table form. I would like to achieve Option 2 output in Option 1 (as this option allows me to gather font, size and other information, performs well for other languages). How can I achieve this ?
   1. table output for option 1 which is good -
   2. table output for option 2 which is bad -
   3. Chinese document text misplaced in Option 1 -
   4. Chinese document text looks good in Option 2 -
   

3. Hyperlinks underline being too long. Is there a way I could resolve this case also ? I think this is happening because of the two reasons - the font being used is not the same in the output and the translated text is not always necessarily the same size and the input text.
   1.

References -

1. https://artifex.com/blog/translating-pdfs-a-practical-pymupdf-guide#:~:text=Using%20PyMuPDF%20and%20an%20appropriate,click%20in%20your%20PDF%20viewer.
2. Replace the text from pdf saving the structure of the document #3499

Here's my example code -

# Option 1
def custom_redactions_with_dict(input_pdf_file, source_lang, page_nos=None, **kwargs):
    import pymupdf
    input_doc = pymupdf.open(input_pdf_file)
    output_pdf_file = input_pdf_file.replace('.pdf', '_translated_custom_redactions_with_dict.pdf')
    translator = <translator_service>

    for input_page in input_doc:
        if page_nos and input_page.number not in page_nos:
            continue
        links = input_page.get_links()
        text_on_input_page_with_bbox = input_page.get_text("dict")
        for block in text_on_input_page_with_bbox["blocks"]:
            for line in block["lines"]:
                for span in line["spans"]:
                    text = span["text"]
                    bbox = span["bbox"]
                    color = pymupdf.sRGB_to_rgb(span["color"])
                    rect = pymupdf.Rect(bbox)

                    translated_text = translator.translate(text, source_lang)
                    translated_text = f"<span style='color: rgb{color}'>{translated_text}<span>"
                    input_page.add_redact_annot(rect, text="")
                    input_page.apply_redactions()
                    input_page.insert_htmlbox(bbox, translated_text)

        for _l in links:
            input_page.insert_link(_l)
        input_page.clean_contents()
    input_doc.subset_fonts()
    input_doc.ez_save(output_pdf_file)

# Option 2
def custom_redactions_with_blocks(input_pdf_file, source_lang, page_nos=None, **kwargs):
    import pymupdf
    output_pdf_file = input_pdf_file.replace('.pdf', '_translated_custom_redactions_with_blocks.pdf')
    input_doc = pymupdf.open(input_pdf_file)
    translator = <translator_service>
    for input_page in input_doc:
        if page_nos and input_page.number not in page_nos:
            continue
        blocks = input_page.get_text("blocks")
        for block in blocks:
            bbox = block[:4]  # area containing the text
            text = block[4]  # the text of this block
            rect = pymupdf.Rect(bbox)

            translated_text = translator.translate(text, source_lang)
            input_page.add_redact_annot(rect, text="")
            input_page.apply_redactions()
            input_page.insert_htmlbox(bbox, translated_text)
        input_page.clean_contents()
    input_doc.subset_fonts()
    input_doc.save(output_pdf_file)

Any feedback would be appreciated.

[JorjMcKie]

I don't have an exhaustive advice, but some single comments:

• I would us the blocks option, because insert_htmlbox dynamically adjusts (shrinks) everything in order to fit. Text length of course changes significantly under translation, so the larger the amount to be written the higher the chance that translated text size stays in a similar order of magnitude.
• That "ff" problem is probably a Story (and hence a MuPDF) issue: it looks for the ligature "ff" and doesn't find it in the chosen font. Maybe it's not even there - it is probably the small font Charis. Recommend to force using a large font via css="* {font-family: sans-serif;}". Or even one of the pymupdf-fonts.
• The link problem must be solved via programmatically inserting links in HTML syntax in the English text. Anything else is bound to fail I think. Similar is true for tables: must be HTML syntax or your effort goes overboard.

[mgalvank]

Thanks a lot for your reply.

1. You suggestion for "ff" problem didn't work for some reason. Can I/we open a bug for this issue in pymupdf ?
2. I decided to go with the block approach. I was able to solve the link and table problem and it was a struggle.
   • Table - using find_tables() I detected the tables, removed the text from the tables. translated the page using blocks. translated the table data using dict and put it back in place, hence maintaining the spacing. Looks good.
   • Link - I had to iterate over all the links on the page, find underlying text, translate it. Store it. Translate text on the page. Find the translated text on the page, add stored link in the new place and update the text with html tags - underline and font color blue to signify it's a hyperlink. Removing old underlines/annotations was painful until I figure out you can redact a location and remove all drawings/images but still keep text in place. This solution is not robust as I am missing some hits but it will do for now. Having the ability to choose the color for underline_annot and highlight_annot would probably be a better alternative.

My main problems are now two -

1. How to reduce the size of the pdf.

   • I do have page.clean_contents() and doc.subset_fonts() where ever necessary but the pdf size is still huge. These two functions were giving me good results before but now with recent modifications they're not giving me good results. I was able to get better results using doc.ez_save()

2. How to maintain the font.

   • Using the block approach, I am missing the font completely. I am just not sure how I could reliably fill in the font and color with this approach.

Btw I cannot not fathom what I would've done if I didn't have this amazing package ! Thanks again for maintaining it and being so proactive !

[JorjMcKie]

.ez_save() is almost the top compression option. It is the same as normal save - just with a handful of different defaults: garbage=3, deflate=True, use_objstm=True. On rare occasions more might be possible using garbage=4, because this option also goes hunting for duplicate object stream contents - sacrificing runtime for it.

[JorjMcKie]

Otherwise, you can use full HTML / CSS syntax power. So you can define a CSS style sheet that defines any font you like for whatever purpose / situation present in the text (= HTML source). Use @font-face/font-family as appropriate.
I would be interested to see whether that ligature problem goes away when a richer font is used.
BTW, Please don't forget that the scope complexity is your own decision ... 😎

[JorjMcKie]

Look into the examples / recipes for .insert_htmlbox() in the documentation to see how it works with a bunch of own fonts.

[JorjMcKie]

You can always open a bug of course.
The only complication is that (1) it is a MuPDF bug if any (2) the MuPDF team requires a bug submission that needs no Python to reproduce. Either C or JavaScript is acceptable because MuPDF is written in C and has a built-in JS interface.
So you need to write a JavaScript that uses a story and exhibits that ligature problem.

[mgalvank]

Hello again.

1. I was able to fix the ff issue as per your suggestion. I used font-family monospace instead.
2. I was also able to get the font style, font color back by using block and dict information together.

But now I am struggling with the font-size.

1. First-of-all, font size inside dict data under span is in points. I need to do size * 1.33 to convert it to px ?
2. I am using insert_htmlbox method to add text to the pdf. If I don't provide any styling but just pure text, the text fits the whole bbox. But if I provide styling, the text size is reduced by a lot. I know there seems to be something wrong with my styling but I just cannot figure out what is going wrong.

Without styling code -

text = "Presidency of the Republic"
page.insert_htmlbox(bbox, text, css=None)

With styling code -

# using random font-size value for testing
text = '<p p_id="0" style="width: 100%; box-sizing: border-box; text-align: justify; font-size: 33.332499999999996px"><span s_id="0_0" style="color: rgb(127, 127, 0);font-family: monospace;font-weight: bold;">Presidency of the Republic</span>
</p>'
page.insert_htmlbox(bbox, text, css=None)

1. Without styling
2. With styling

What I would like to be the outcome ->
I do not have to provide the font-size. The text automatically fits the size of the htmlbox just like it does when no styling is provided.
Any feedback is appreciated ! Thanks.