PDF page to image conversion on upload - #121
Conversation
|
|
||
| redirect_resource do |resource| | ||
| resource.image? ? resource.content_converted_iiif_url(page_number) : resource.content_iiif_url(page_number) | ||
| if resource.converted_pages? |
There was a problem hiding this comment.
Do the other endpoints in this file need the resource.converted_pages? check and fork to use the new URLs you defined in resource.rb?
e.g. image_api can use content_converted_pages_image_api_url, info can use content_converted_pages_info_url, same sort of thing for preview and thumbnail
There was a problem hiding this comment.
Good point; I've added all the ones that seem relevant but I admit to not having a thorough sense of all the contexts in which these endpoints are used so I haven't tested fully. Thumbnails and previews seem to work as expected at least.
There was a problem hiding this comment.
info: redirects to info.json, the IIIF Image Information doc (dimensions, tile sizes, formats). Viewers fetch this once before requesting any tilesimage_api: the full IIIF Image API endpoint (with params like size, rotation, etc). This is the most-fetched endpoint by IIIF viewers with deep zoom capabilitiesiiif: convenience redirect to an image or page as a JPEG (at full max res/full/max/0/default.jpg)previewandthumbnailare pretty self explanatory, they are fixed size JPEG derivatives
|
|
||
| redirect_resource do |resource| | ||
| resource.image? ? resource.content_converted_info_url(page_number) : resource.content_info_url(page_number) | ||
| if resource.converted_pages |
There was a problem hiding this comment.
| if resource.converted_pages | |
| if resource.converted_pages? |
typo?
| redirect_resource do |resource| | ||
| resource.image? ? resource.content_converted_info_url(page_number) : resource.content_info_url(page_number) | ||
| if resource.converted_pages | ||
| resource.content_converted_pages_info_url(page_number) |
There was a problem hiding this comment.
Should we do page_number.to_i everywhere the new helpers are called, since it does a numeric comparison after passing the arg to content_converted_pages_base_url?
|
|
||
| redirect_resource do |resource| | ||
| resource.image? ? resource.content_converted_iiif_url(page_number) : resource.content_iiif_url(page_number) | ||
| if resource.converted_pages? |
There was a problem hiding this comment.
info: redirects to info.json, the IIIF Image Information doc (dimensions, tile sizes, formats). Viewers fetch this once before requesting any tilesimage_api: the full IIIF Image API endpoint (with params like size, rotation, etc). This is the most-fetched endpoint by IIIF viewers with deep zoom capabilitiesiiif: convenience redirect to an image or page as a JPEG (at full max res/full/max/0/default.jpg)previewandthumbnailare pretty self explanatory, they are fixed size JPEG derivatives
2 participants
In this PR
An initial attempt at improving the performance of large PDFs by processing the individual pages into TIFF files on initial upload, and referencing those files in the generated manifest. Specifically, it makes the following changes:
pages_countfield to theResourcemodel for bookkeeping (NOTE: this is not currently relied on for anything, but it's not a lot of data and seems like it can't hurt to make this data easily accessible);has_many_attached :content_converted_pagesattachment type to the Resource model, along with some related methods; this allows us to store multiple pages as related to a single parent PDF object.convert_pdfservice that loops through the individual pages of the PDF and converts to TIFF.alphachannel that the JPEG compression couldn't handle. Adding- alpha offto the conversion seems to have fixed that error.create_manifest_jobto use the page-specific image links for the items, rather than relying on the;{page_number}suffix.I should also mention that to get this working on my machine, I had to install
ImageMagickandGhostscript(I think that's what it was called?), which are used for parts of the PDF processing. Possibly an update to the Docker script to make sure those are included on the server would also be necessary, if they're not there by default.Disclosure: Some of this was done with the help of the copilot chat thingy in VSCode, for help with proper Ruby syntax and some initial suggestions about how to incorporate the multi-page PDF concept into the content model, and handling some of the new URL methods. I've been through all of the code (I had to, since a lot of it did not work...) and I've gotten it to the point of successfully processing a PDF in my local environment and showing it in the IIIF viewer with the pages pointing at the newly generated TIFFs. As mentioned I have some lingering concerns about the quality of the individual page images, and also we would need to very thoroughly test corner cases and try it out with a wide range of PDFs. We should also see if we can actually quantify any performance improvements over the old method. But it's a start. ^_^;