Adding pdf/epub/mobi generation from the markdown files using pandoc by holgern · Pull Request #67 · btcguide/btcguide.github.io

holgern · 2021-01-14T08:23:04Z

This PR adds the necessary files to generate a pdf, epub and a mobi document from the markdown sources.
It uses a template for latex and epub generation (assets/templates). batch and bash scripts to generate pdf/epub have been added. I also explained what to install to the readme file.
Instead of index.html a new main index_pandoc.md is used as main file.

lua filters are used to do the following:

remove the yaml header from each markdown file and convert it into a markdown header
Fix the image path
Fix the width and class parameter which cannot be handled by pandoc
Fix the internal links, as pandoc cannot use the filename as label
Load markdown files in include
Adjust section levels (header levels cannot be skipped)

I added also a github action that is run everytime a new PR is merged or something is added and does the following:

creates a new Tag with the current date
Creates a pdf/epub/mobi file and upload it as release

You can view how it will look like here:
https://github.com/holgern/btcguide.github.io/releases

I'm not a LUA expert (I learned LUA with this project), so the lua code is not optimal, but works :).

The pandoc github action is failing for this PR, as it cannot create a tag yet.

* Images width has been increased * lists have been fixed * header levels have been fixed

… into btcguide-master

merge upstream

….github.io into btcguide-pandoc

… into btcguide-pandoc

merge upstream

…perrefs

Merge upstream changes

mflaxman · 2021-01-18T13:54:52Z

Thanks @holgern! Been getting a lot of PRs on the guide so I haven't yet given this the attention it deserves but it's really cool.

I want to get it working with GitHub so the script automatically generates the downloadable versions when content is updated.

mflaxman · 2021-01-18T23:35:10Z

Hey @holgern, I spent some time playing around with this and it's pretty neat! I also cleaned up the readme/script a little bit:
https://github.com/btcguide/btcguide.github.io/tree/pdf-attempt

It appears the #s produced by this script are kind of messed up unfortunately:

I played around with it for a while, but I wasn't 100% sure I'm following the lua code.

mflaxman · 2021-01-18T23:35:46Z

_pandoc_filter/add_title.lua

+end
+
+
+function skip_include(el)


Extremely confusing name, either rename to something that makes sense or add a comment

mflaxman · 2021-01-18T23:36:27Z

_pandoc_filter/add_title.lua

+	        element.src = fix_path(element.src)
+            return element
+		   end
+		   })


IDK lua, but are there usually standards for indenting? This is all over the place.

mflaxman · 2021-01-18T23:37:36Z

_pandoc_filter/add_title.lua

+	   end
+	   doc = pandoc.read(content)
+	else
+	   io.stderr:write("Warning: --- was not found twice at: " .. title .. "count: " .. count)


The errors this prints out appear to be nonsensical:

pandoc --pdf-engine=xelatex --template=assets/templates/eisvogel.latex --highlight-style zenburn --toc -N --lua-filter _pandoc_filter/image_link.lua --lua-filter _pandoc_filter/add_title.lua -o assets/btcguide.pdf index_pandoc.md ./assets/img/setup-specter-detect-node.png ./assets/img/setup-paper-calculate-seed.png [WARNING] Duplicate link reference '[TODO]' at line 26 column 1 [WARNING] Duplicate link reference '[TODO]' at line 24 column 1 hw/python.md hw/shitcoins.md hw/encouragement.md hw/psbt.md hosted/utxo_privacy.md hosted/utxo_privacy.md hw/python.md hw/psbt.md hw/psbt.md hw/wired_airgap.md

mflaxman · 2021-01-18T23:38:35Z

_pandoc_filter/add_title.lua

+
+end
+
+function filter_content(content)


Is there no way to DRY this out? This jekyll has already gotten far too complex for what it is :(

That's partially my own fault :(

mflaxman · 2021-01-18T23:38:53Z

_pandoc_filter/add_title.lua

+    content = string.gsub(content, '%(advanced#redundant_address_verification%)', '(#verify-receive-address-advanced)')
+	content = string.gsub(content, '%(/backup%-wallet%)', '(#backup-wallet)')
+	content = string.gsub(content, '%(/backup%-wallet/seeds%)', '(#backup-seeds)')
+	content = string.gsub(content, '%(/backup%-wallet/public%-keys%)', '(#backup-public-keys)')


Can this be DRY?

mflaxman · 2021-01-18T23:39:33Z

_pandoc_filter/add_title.lua

+function Para(elem)
+    if #elem.content == 1 and elem.content[1].t == "Image" then
+      local img = elem.content[1]    
+      if img.classes:find('markdown',1) then


Can this be DRY?

Also, unclear what markdownskip and these #s are (might be clear if DRY).

holgern · 2021-03-12T22:49:59Z

I'm still interested in proceeding.
I will go over the LUA script and try to improve it.

Thinks could be simplified when the header levels would be fixed directly in the markdown code.
e.g.

##

####

should be fixed to

##

###

Should I make a new PR in which the header levers are fixed so that the LUA script could be simplified?

merge upstream

mflaxman · 2021-03-23T18:44:44Z

Should I make a new PR in which the header levers are fixed so that the LUA script could be simplified?

Sorry, I'm not following what this change would do, can you give me an example page of what the before/after would look like?

Is the idea that I'm jumping from 2 #s to 4 #s and I should be going from 2 #s to 3 #s? It's hard to evaluate without an example, but generally I'm trying to avoid large text headers when they aren't needed.

holgern and others added 30 commits December 10, 2020 13:15

Initial release

8c60b59

Improved formatting

7c424a6

* Images width has been increased * lists have been fixed * header levels have been fixed

Add create_pdf.sh

4205d15

Merge branch 'master' of https://github.com/btcguide/btcguide.github.io…

b2acae3

… into btcguide-master

Merge branch 'btcguide-master' into btcguide-pandoc

bdb3e22

Merge upstream changes

91d983e

Improved structure and bug fixes

40bc961

Use lua filter to correct image links

b170e34

Use lua filters and change source files as little as possible

e844f6b

Merge pull request #3 from btcguide/master

2ac3dbb

merge upstream

Add gitlab ci to create pdf file automatically

d303583

Create main.yml

7a89865

Delete .gitlab-ci.yml

2954334

Merge branch 'btcguide-pandoc' of https://github.com/holgern/btcguide…

7fce203

….github.io into btcguide-pandoc

Rename github action and add epub creation

dcfeffc

Change action name

76697f7

Add create release to ci

a57bf57

fix typo

42354f5

Change github token name

673702f

Update pandoc.yml

6b5235a

Use date as release name

55896bf

Add tag name

495c648

Create tag and push release

49f65da

Change list

d340e92

Update tag

a64fc15

Fix tag name

30ac0a8

Merge branch 'master' of https://github.com/btcguide/btcguide.github.io…

43861e8

… into btcguide-pandoc

Fix internal links in pandoc

67fdff3

Add mobi file

1f9cab6

Update pandoc.yml

b7a08e0

holgern and others added 10 commits January 13, 2021 11:23

Fix pandoc creation and remove btcguide docs from repo

2296832

fix typo

a49179d

Try to improve mobi outpout

6eb475e

Fix command and improve readme

f2de589

Fix spellchecker and improve readme

06afe1e

Rename index.md to index_pandoc.md and fix spellcheck

dcc2ba4

Merge pull request #10 from btcguide/master

af75506

merge upstream

Fix header generation with correct hyperrefs and resolve duplicate hy…

0197830

…perrefs

Move template folder into assets

95dbdf1

Merge pull request #11 from btcguide/master

af9552a

Merge upstream changes

mflaxman pushed a commit that referenced this pull request Jan 18, 2021

Fix list items to work better with #67

98a9797

mflaxman requested changes Jan 18, 2021

View reviewed changes

mmikeww pushed a commit to mmikeww/btcguide.github.io that referenced this pull request Jan 31, 2021

Fix list items to work better with btcguide#67

3549501

Merge pull request #13 from btcguide/master

ebc42ae

merge upstream

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding pdf/epub/mobi generation from the markdown files using pandoc#67

Adding pdf/epub/mobi generation from the markdown files using pandoc#67
holgern wants to merge 41 commits intobtcguide:masterfrom
holgern:btcguide-pandoc

holgern commented Jan 14, 2021 •

edited

Loading

Uh oh!

mflaxman commented Jan 18, 2021

Uh oh!

mflaxman commented Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Uh oh!

holgern commented Mar 12, 2021

Uh oh!

mflaxman commented Mar 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

holgern commented Jan 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mflaxman commented Jan 18, 2021

Uh oh!

mflaxman commented Jan 18, 2021

Uh oh!

mflaxman Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

mflaxman Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

mflaxman Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

mflaxman Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

mflaxman Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

mflaxman Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

holgern commented Mar 12, 2021

Uh oh!

mflaxman commented Mar 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

holgern commented Jan 14, 2021 •

edited

Loading