Hi! I'm a bit new to this space, but something tells me that I might stick around even after this project.
As an aspect of my capstone assignment, I've been tasked with implementing an extension for Inkscape utilizing the OCR program (a program that converts an image into text) Tesseract to convert handwritten words into text. It is under the Apache 2.0 License so there are no real issues with me using their code. I need to be able to extract handwriting from an entire SVG file and from user selected features, and either replace those features that are detected to be words with a proper text box or dump them into a separate text file. The former would be accomplished with an effect extension, the latter an output extension
I expect to accomplish the entire document extraction by using the already existing export functions in Inkscape to convert the SVG into a raster image and feeding that through Tesseract, but I am not sure if its possible to rasterize only selected features. I also only have vague ideas on how to actually replace elements.
Overall I think I can get this done in the proper timeframe, but what I really need is a good introduction to extensions in Inkscape. I've been looking at the tutorials Here, I've looked over the resources on the main website, and I already have a background in C++ and Python. It looks like the tutorial linked has templates but I do not know where they got it and it does not provide a template for the an output extension.
I will most likely be back later with more specific questions, but I just kinda wanted a preface I can link back to for my inevitable future questions. Thanks y'all.
Yeah okay this is a. pretty dumb one. I feel like I've made some good progress, but I'm having a bit of an issue with something that feels really simple. One of my goals is to just funnel an entire .svg into Tesseract, but in order to do that I have to convert the document to a PNG first. I thought I had figured out how to do this, but turns out I clearly... do not know what I'm doing 😅
A
class DocumentToTXT(inkex.RasterOutputExtension):
def load(self, stream): from PIL import Image
# disable the PIL decompression bomb DOS attack check. Image.MAX_IMAGE_PIXELS = None
self.img = Image.open(stream)
def save(TextToSave): TextToSave = pytesseract.pytesseract.image_to_string(self.img) #Insert save dialogue things here
would do the trick, but it just leads to the error message
Which, if I'm reading it right, is trying to send the .svg to PIL, not a PNG. I had figured it seemed too simple, but I haven't been able to figure out why. I figure part of my mistake is mostly likely related to a misuse of RasterOutputExtension, but I get the same issue when putting in EffectExtension, so while I'm still probably using it wrong its not the reason I'm getting this particular error.
"Takes a PNG from Inkscape and outputs it to another rather format."
I've never used this. From what I can see, it does not do what you think it does.
It seems to be just a method for saving an existing raster in Inkscape to another format using PIL.
This would be either a linked or embedded (base64) existing <image>
--
Converting svg>raster is something commonly done, and there are several methods to do it.
Browsers do it, gdkpixbuf can do it, and other command line untilties.
So why does anyone use Inkscape to convert svg>raster ?
It's because of the rendering accuracy. This is particularly important for text rendering.
In Inkscape to convert svg>raster in the extension system you have two choices:
1. Use one of the more inaccurate non Inkscape methods. Which may be good enough. You do not need any external modules for this if you are using Inkscape 1.1+ (except on macos I think)
2. Use an Inscape command call which effectively calls Inkscape invisibly from the command line.
Number 2 has a speed penalty and requires the use of a temporary file.
You can minimise the speed reduction by trying to fit everything into a single command call.
That... would explain a lot. Thank you for the information. Do you know where I can find documentation on inkscape's command line functions? I'm aware I can most likely just use ---help but a file or documentation page would be useful as well
...I'll be honest, this was shamelessly ripped from this previous post on the forums. It doesn't seem to want to output any text, despite what I try. My string does have newlines, which was mentioned in that previous post to be finicky, but I get the same problem even when I set txt.text = 'test'
Yet again I ask for the wisdom of my elders for a problem I presume nobody else has 😅. Thank you for your patience with me
Creating New Extensions Tesseract OCR
Inkscape.org
Hi! I'm a bit new to this space, but something tells me that I might stick around even after this project.
As an aspect of my capstone assignment, I've been tasked with implementing an extension for Inkscape utilizing the OCR program (a program that converts an image into text) Tesseract to convert handwritten words into text. It is under the Apache 2.0 License so there are no real issues with me using their code. I need to be able to extract handwriting from an entire SVG file and from user selected features, and either replace those features that are detected to be words with a proper text box or dump them into a separate text file. The former would be accomplished with an effect extension, the latter an output extension
I expect to accomplish the entire document extraction by using the already existing export functions in Inkscape to convert the SVG into a raster image and feeding that through Tesseract, but I am not sure if its possible to rasterize only selected features. I also only have vague ideas on how to actually replace elements.
Overall I think I can get this done in the proper timeframe, but what I really need is a good introduction to extensions in Inkscape. I've been looking at the tutorials Here, I've looked over the resources on the main website, and I already have a background in C++ and Python. It looks like the tutorial linked has templates but I do not know where they got it and it does not provide a template for the an output extension.
I will most likely be back later with more specific questions, but I just kinda wanted a preface I can link back to for my inevitable future questions. Thanks y'all.
Yeah okay this is a. pretty dumb one. I feel like I've made some good progress, but I'm having a bit of an issue with something that feels really simple. One of my goals is to just funnel an entire .svg into Tesseract, but in order to do that I have to convert the document to a PNG first. I thought I had figured out how to do this, but turns out I clearly... do not know what I'm doing 😅
A
class DocumentToTXT(inkex.RasterOutputExtension):def load(self, stream):from PIL import Image
# disable the PIL decompression bomb DOS attack check.Image.MAX_IMAGE_PIXELS = None
self.img = Image.open(stream)def save(TextToSave):TextToSave = pytesseract.pytesseract.image_to_string(self.img)
#Insert save dialogue things here
would do the trick, but it just leads to the error message
PIL.UnidentifiedImageError: cannot identify image file <_io.BufferedReader name='/tmp/ink_ext_XXXXXX.svgGI7VK1'>Which, if I'm reading it right, is trying to send the .svg to PIL, not a PNG. I had figured it seemed too simple, but I haven't been able to figure out why. I figure part of my mistake is mostly likely related to a misuse of RasterOutputExtension, but I get the same issue when putting in EffectExtension, so while I'm still probably using it wrong its not the reason I'm getting this particular error.
TL;DR: How do I raster my SVG file mid-extension?
According to the inkex API documentation:
https://inkscape.gitlab.io/extensions/documentation/source/inkex.extensions.html?highlight=raster#inkex.extensions.RasterOutputExtension
"Takes a PNG from Inkscape and outputs it to another rather format."
I've never used this. From what I can see, it does not do what you think it does.
It seems to be just a method for saving an existing raster in Inkscape to another format using PIL.
This would be either a linked or embedded (base64) existing <image>
--
Converting svg>raster is something commonly done, and there are several methods to do it.
Browsers do it, gdkpixbuf can do it, and other command line untilties.
So why does anyone use Inkscape to convert svg>raster ?
It's because of the rendering accuracy. This is particularly important for text rendering.
In Inkscape to convert svg>raster in the extension system you have two choices:
1. Use one of the more inaccurate non Inkscape methods. Which may be good enough. You do not need any external modules for this if you are using Inkscape 1.1+ (except on macos I think)
2. Use an Inscape command call which effectively calls Inkscape invisibly from the command line.
Number 2 has a speed penalty and requires the use of a temporary file.
You can minimise the speed reduction by trying to fit everything into a single command call.
That... would explain a lot. Thank you for the information. Do you know where I can find documentation on inkscape's command line functions? I'm aware I can most likely just use
---helpbut a file or documentation page would be useful as wellhttps://inkscape.gitlab.io/extensions/documentation/source/inkex.command.html
| and either replace those features that are detected to be words with a proper text box
Hey so about this part. I have been trying to actually insert a textbox, but i'm finding difficulty doing so right now.
xcoord = str(blockLeft)ycoord = str(blockTop)
txt = inkex.TextElement(x=xcoord, y=ycoord)
txt.style = 'font-family: serif; font-size: 36pt'
tspan1 = inkex.Tspan()
tspan1.text = 'bold'
tspan1.style = 'font-weight: bold'
txt.append(tspan1)
txt.set('xml:space', 'preserve')
txt.text = blockText
...I'll be honest, this was shamelessly ripped from this previous post on the forums. It doesn't seem to want to output any text, despite what I try. My string does have newlines, which was mentioned in that previous post to be finicky, but I get the same problem even when I set
txt.text = 'test'Yet again I ask for the wisdom of my elders for a problem I presume nobody else has 😅. Thank you for your patience with me
Hello Claire,
I don't know what is the status of your project, I did my own extension here : https://inkscape.org/forums/extensions/ocr-extension/
(also based on tesseract, by i'm not using python module, just expect a tesseract installation available in PATH)
Franklin