Reorganizing my blogging experience – write markdown in Evernote with Postach.io and sync to WordPress

Original post: http://kleineblase.postach.io/reorganizing-my-blogging-experience-write-markdown-in-evernote-with-postach-io-and-sync-to-wordpress

I loved Postach.io the first time I saw it in Evernote Dev cup this year. I know it will win and it did.
As I was also a participant, I know how smart they have to be.

Put it simple, you write in evernote and it will automatically publish to your blog.

highlight is it solved two problems at once
– you want to save what you wrote really secure – Evernote did it
– you want to keep the posts updated – you update once in Evernote and it publishes & sync to the audience

TO-BE Workflow

Nth explain better than a picture

Workflow

It looks complex, but good news is most paths are just automated and I focus on the main part – content.

Requirements and old issues

Several things I need when I blog:

Secure, available, anywhere

Availability is why I don’t host my own blog like other developers do. I have enough things to busy with and maintaining a server up is not in my list.
I am fine with WordPress’s free account. At least I know my blog is unlikely to be down.
Just after all blog providers do shut down, like xanga. There are more concerns like encoding and you want to have control on your own stuff and backup them.
Lost everything during you have written 80%, I tried, is just a terrible experience. Find a reliable editor.

Evernote is very secure in all these which has goal to keep it for at least 100 years

Write once, update everywhere

It is most important. In my existing workflow, I write in evernote or other editors then publish to wordpress. Then I find a grammar mistake. I find a link broken. I need to update that in local and then edit that in the duplicated wordpress again. Make things worse, formatting matters. I used markdown – I need to convert manually, healthcheck and post again. This is the least fun part of blogging and kills productivity.

Markdown and formattings

Markdown is one of the greatest “discovery” about formatting text.
I just cannot hate any kind of text-formatting methods any more. WordPress has a nice WYSIWYG editor – but it is still annoying when you blog.
Not to mention these formats are very unlikely to transfer seamlessly among different platforms. e.g. your Word editor and on WordPress. Not to mention when you have code snippets to show.

When blogging, keep source in markdown. Publish the converted markdown is much easier.

I spent quite a lot of effort to investigate how to use it everywhere and with its traction I know many people have the same question.
I also created Cheeatz on how to covert markdown & gist in evernote, for developer cheeatsheet purpose.

Platform and Audience

SEO and analytics of wordpress is really nice. You know people from which country visited, which keywords they used in search engine to reach your blog. Mobile app is also available.
This is something postach.io unlikely to beat in near future. There are also communities and social networks within wordpress then make your blog higher exposure.
like the “ping back” system – your posts are visible in posts you mentioned, and they know you mentioned them.

Medium is a high-profile blog platform startup that I am interested. (By Evan Williams who is “Making systems for typing and thinking” (twitter/blogger)). However, so far I don’t see killer features.

Postach.io is able to auto-share to twitter and Facebook which is good enough for me. WordPress there are more hooks, up to your need. More sophisticated user may want to even control the time of publish – to avoid posts ignored by social network algorithms when finish it tiredly at 3am.

My take will be – keep the wordpress there for audience and perhaps availability. I will sync from postach.io to WordPress.
This is tricky – syncing directly from evernote to wordpress is doable via tools like zapier.

Proofreading

When I blog, I am in hype. You are excited about your ideas and you want to push it right away. After publish you feel a sense of “done” meanwhile you keep checking has anyone commented.
One day or two after you found it tastes a bit different, you got other thoughts and you can see there are quite a lot of grammatical mistakes.
It is how our brain functions – big topic aside – review is better done a while later after your are less “dived” in details.

====

So above are my requirements, below are my actual flow ( refer to the diagram)

Markdown editors

Writing markdown directly in evernote may not be that friendly – you want preview.
This has been discussed several times – one can use online editors Markable.in or Stackedit(https://stackedit.io/) or my own trial cheeatz.com which, to confess, is little bit broken now.
I am trying to write in sublime – with syntax hightlighting and then use SublimeEvernote(https://github.com/jamiesun/SublimeEvernote) to send to Evernote. This is not perfect, as every time a new note is created.

Sync New post in postach.io to WordPress

Zapier or IFTTT are great services to schedule automated tasks among your cloud services. Meanwhile Postach.io is not available in Zapier yet.

Sync directly from evernote to WordPress has an issue – there is no conversion for markdown in source.

Luckily, we can use RSS postach.io which shows list of entries in postach.io, so

New Item in Postach.io RSS -> Create new WordPress Post

You should find the RSS url in your postach.io blog (depends on your template) and you can configure the RSS feed in settings.
e.g. mine is http://kleineblase.postach.io/feed.xml

I created a simple recipe here, not sure if it works for you

Use “Content Text”, not Content directly

With this whenever I blog in postach.io, a wordpress entry is created!
It looks good, as at least wordpress is able to feed in the text with format in html.

Improvement and Tradeoff

Posts update in postach.io are not sync to wordpress

I am not a hollyword star so managing feedbacks in multiple platforms is not my top concern. I focus more on the workflow of producing. (an idea will be try to use same disqus in different platform? I am not sure, free wordpress don’t support this anyway)
It is also less likely to need to update wordpress if original post went through proofread process.

This is little big hard – Need postac.io ‘s RSS feed to provide information of “Update” or a smart tool that do more to keep removing & updaing rss items.

Anyway, the current approach is good enough to keep my workflow running

Create Preview in postach.io

There seems to be no draft preview in postach.io
One trick is to have a “pre-prd” /”staging”, well, a less geek term, draft notebook which corresponding to a private blog and you (and your peers!) can review it before actual post. It is not possible to make it totally private, but I think for me it is fine. You feel a fake sense of “done” meanwhile you can ensure its quality.
To switch from “pre-prod” to “prod” (actual blog) is just a notebook switch in evernote.
Btw i just did proofread and publish of this article using evernote app in mobile and I find it comfortable.

All in all

I am experimenting with the workflow. Postach.io is not very mature, but I think it will grow and give us more features.
(I can see some linked produced by my markdown syntax []() breaks if the link is picked up by evernote already. Ironically, when you “paste as plain text” in Evernote, it is automatically converted into link which I need to remove manually)

I also did not disucss on one topic – how to organize your “to blog” ideas, which need further explore. Without suprise before I blog I had some ideas written in —Evernote, which I will tag with “to blog”. I also create lists in Trello for things I need to write.
Also there are some work to export my old wordpress blog posts to postachio, which here is a reference

Hope this post helps you. Credits, if any, should be given to above tools’ teams and most importantly, Aaron Swartz, who invented RSS and markdown.

Advertisements

Reorganizing my workflow and “info flow”

Original post: http://kleineblase.postach.io/reorganizing-my-workflow-and-info-flow

This is the first post in this postachio blog, and first of series of posts – as titled.
I am obsessed in optimizing my workflow. I am also obsessed in information retrieval.I read wikipedia for consecutive several hours. I keep distracted by articles. I do “research” from astronomy to chinese medicine while I consider my job a developer who naturally read tons of documentation & code.  Everyday I am fed up with information and feel there are tons of things that are outstanding.I need to organize them and finally I have time to work on it.
I am talking about cognitive limit of human here – re-organizing this not only make me more productive, it reduces my cognitive stress. It definitely changes my brains as the Internet does
This is the outline of stuff. I dare to consider myself like a machine which do “IO”.
Input:– articles- books- ideas
workflow– tasks- calendar
output:– blogging- notes- (social network)
The first thing will be of course, about blogging. Creating this new blog is part of it.

Code for Hong Kong

tl;dr

I am founding a team & an organization – Lets gather geeks to hack for a better hong kong and meanwhile sharpen our skills

Why

It all starts with Code for America

Jennifer Pahlka: Coding a better government is one of the best TED talk I ever watch. It is an initiative that creates network of geeks, act to speak louder than words, shows what is possible and mobilize citizen engagement to change the government. Rock star developers are hired to work as an internship in government and revolting the bureaucracy using technology, encouraging citizen participation in the governing process.

I can’t explain better than the 12min tech talk. Please really watch the video and visit their website especially if you consider yourself a technology professional.

Code for Hong Kong?

There is Code for America. Taiwan folks asked can they have Code for Taiwan and created g0v.tw. Numerous hacktivists around the globe work hard to shape our world.

Hong Kong need this more than any other places. Hong Kong is a great city, just now sick. Anyone with common sense will know Hong Kong is having problems in numerous perspectives which need fundamental changes in how our government and society works. Every year Hong Kong people march to voice out our opinion and anger. I see talents go away as Hong Kong get worse.

Technology is one of the few things that stands a chance to fight against monopoly and bureaucracy. Egypt succeed in evolution with the help of social media technology. Startups grow rapidly in Chile or Vietnam changing the landscape. After empty words of “Cyberport” and the dot.com bubble years ago, we need to ask why we didn’t have technology revolted our lovely Hong Kong. We need to show the possibilities. We need to engage developers and every citizen to participate in saving our city. More importantly we need to TRY.

Your lack of software and science start-ups will hurt you

Eric Schmidt commented that on Hong Kong’s technology industry during his recent Hong Kong visit . It is about capability, having a strong technology industry to support social movements. It is also about respect. As the Chinese saying goes, a man must despise himself before others. The first thing to do for gaining respect is respecting yourself. There are all sorts of developers out there. The industry is largely misunderstood and people can’t differentiate good from bad and we are hurt as a whole. There must be reasons why most other countries software developer enjoy easily twice the wages and higher social status. It is our own responsibility to build up the industry such that it is as professional as when people talk about doctors, lawyers or accountants. Hong Kong need developers that knows their stuff and shows professionalism. We are talking about Ethics here — The book Clean Coder is the bible for programmers on this. Good news is we can improve together and community is growing. Start-up incubators, social entrepreneurship facilitators and co-working spaces are now there.

How

Another organization?

There are nice communities out there and I am glad to have joined them this year. To name but a few, CodeaholicsHK Web developers Meetup and Open Data Hong Kong are awesome. I also see lots of passionate developers or designers that start up ideas which are turning Hong Kong into a better place. From LegcovotesLawn Map tohttp://www.diamondcab.com.hk/

I myself as a developer faced lots of difficulties when I try to code something like that. These ideas need continue support after hype in hackathons. Adding features, keep servers running or gain media coverage. These need to be done after overtime for your day job.

Code4HK want to unite these initiatives and work as an engine to support them. We seek practical ways such that we together achieve the cause.

Guild as the answer

How such organization should work, guild is perhaps the answer. After inspirations from Spotif’s Tribes/Squads/Chapters & Guild.

Guild is an old concept. For us it resembles our experience in online games like World of Warcraft. Meet friends, beat the bosses together, have fun and level up.
Of course it actually exists since the Middle Ages for craftsmanship. It is where apprentice can become master. Every programmer have their own job. Their own projects. Their own skills and corresponding community. Code4HK as a guild serves to co-ordinate them for a common cause.

There are communities sharing the same cause of coding a better world – Social Innovation Camp , Open data HK or #hack4good. We are trying to position to complement them and look for their support as well. We hope members form teams bringing our ideas to their hackathons and hack there. Members join communities sharing our ideas and exchange technologies. Through these initiatives we achieve our 2nd goal – peer improvement

Peers

We don’t want a hierarchical organization. Everyone can code for Hong Kong. Everyone can become a “crew member” and form teams. These teams get their hands dirty and work on “missions” as open-source projects that will change Hong Kong. Again, we position these teams as SWAT that act and deliver. Code jam (Dojos) are organized for peers to hack on technical stuff together to know each other better and improve their skills.

What

We are building applications including map that showing pollution spots; app that counts number of participants in 1st July demonstration using big data and scientific way. We want to create HK version of PolitiFact, a crowd-sourcing fact-checker of HK politicians. We want to make platforms allowing citizens to voice out why they march on 1st of July, to vote, discuss on politics issues and organize themselves for protest.

With all these we want to show possibilities and explore how far proxy democracy can go and how citizen engagement and technology will make the difference.

It is possible to run hack days for Charity, helping NGO to setup websites, there are limitless potential meanwhile we will have focus and priority in changing the way Hong Kong government works.

Stay tuned

I already have brilliant friends that take this serious and consider these goals as one of their career objectives. Currently this idea is in incubating stage that we are looking for founding crew of 5-7 people to explore how things work out. Still we will need your support and know how you feel. We hope to deliver the ideas and open it to public soon in 2014. Follow @code4hk and #code4hk in twitter. Leave a comment if you have any idea.

Graph Database in Javascript?

This crazy thought came to me after working on some stuff on graph data structure — Let’s implement a graph DB in javascript

(I always have thoughts on putting everything in javascript, while most of the cases someone did it.)

Why

Such db supporting most basic features, transaction, graph query has following purposes.

  • common data visualization

It is very likely you have some graph-based data visualization at front end (d3.js) and a graph database (neo4j) at the back. What if the graph db can output the graph representation that can easily bind to DOM with adapter and visualized ?

  • cache at client side

It is possible to interact with the graph front end, but for any new query you need to send to the back again

What if we can query a subgraph and store it in the js graph db as a cache and support basic query functions. This greatly reduce traffic, reaction time and Dom binding for data visualizations

  • in javascript

Javascript itself is sexy. Ok aside from being nerd about it — it is easy to code, with nice optimization and when you have everything in javascript (mobile,web,nodejs) you don’t want to have a dependency on backend in java or so. We are talking about the future – this is not targeting to replace neo4j

The prototype should have these characteristics

  • in memory

at the moment it is aimed that the graph model itself is represented using javascript objects itself, to facilitate the visualization and simpler coding.

It is possible to use Node.js or local storage to support persistence, but that is not in current scope as the design will be very different

  • open source

of course.

  • Basic

At the moment, it does not have to scale very nice, high availably, recovery which is the real complex part of database

We are talking about a In-memory, light-weight database in javascript that can run in both browser and nodejs

Most I know is from the concise & inspiring book Graph Database

The most crucial point about graph database in my mind is about index-free-adjanceny.

We need to focus on this fundamental problem. Is this doable in javascript?

Neo4j is the reference architecture for discussion. The Graph DB Internals chapter of the book is a highly recommended.

Just one thing neo4j very different is that neo4j is a file-based storage, while currently we discuss a in-memory database prototype. Still, most concepts apply.

Lessons from Neo4j

Firstly some words on index-free-adjanceny —

Graph traversal is cheap because every node has reference to node it is connected to, forward or backward.

In databases that we usually refer to (mostly RDBMS), we can use index to locate a record. (i.e. when look up by id)

This is cheap – but only relatively. When the record transaction id 1 say it is of customer id 100, you need another index lookup, which is o(n * logn), and you know the long painful stories on optimizing database to get the index right

Index-free-adjanency is about in every node you have the reference, i.e. location of the node that is connected to. i.e. O(1), without doing a search, which even indexed takes O(log n)

Neo4j also put this considerations in the file-based storage layer.

Node store file of neo4j is fixed length (9 byte), makes the lookup by id an O(1) operation. (e.g. with id 100 we know it is at 900bytes)

With this context, traversal in graph is fast – because going from node id=1 is to another node id=99 , we chase the “next id” in a node and just move to the known byte location of it – which likely in the same fixed-length

Although how far this concept goes I am not sure, -Transversal across files/machine? as

“User’s view on the graph and actual records on disk are dissimilar.”

It is also worth mentioning the mathematical problem of optimally partitioning graph across in distributed environment is NP-complete problem.

1st class Relationship

Also in Graph DB like neo4j, relationship itself is a “first-class citizen”

Each Node doesn’t simply store pointer of next node, every relationship is store and allow easy lookup. this facilitate going back and forth in the graph (relationship is directed)

Bear in mind a node can connect to multiple nodes. Pointers are also stored in relationship such that we can go to one side’s node ‘s next or previous relationship cheaply.

Wont go into too much details as most important is the idea. Slide 12 here shows the actually record “schema”

In javascript?

we need to consider the language itself.

Naively, it is possible to use object reference to get objects linked.

For example, with

Node Alice —loves—> Node Bob –loves–> Node Charlie

(Sadly in this world of cryptography characters, there is no one love each other)

Alice = {

 "nextRel": Love_1

}

Bob = {

 "nextRel": Love_2

}

Love_1 = {

 "first_node":Alice,

 "second_node":Bob,

 "relationType": "love"

}

However, this may not be sufficient for graph DB – we need low cost traversal.

If I need to go to next node of a, b, there are two steps here: First get the reference of Bob

Alice[“nextRel”][“second_node”] itself is two object lookup, which is not that cheap apparently.

So what is closer? Array?

One of the very characteristic about javascript as you may know, Array is not really an “array” in sense of other languages.

When we learn C, malloc()

Array are consecutive memory locations which makes it really efficient to lookup value and process together.

In Javascript, arrays are Objects. Arrays can be sparsed.

A equivalent data structure don’t really exists in javascript, but the good news is most javascript engine try to make javascript work like that.

We need to know how far this goes and make sure this optimization in place.

In engines like V8, hidden classes are created behind the scence such that property access don’t require a dictionary lookup.

Furthermore, Array Objects are special in the specification – 15.4

Index-based Arrays are optimized.

A for loop on an index-based array is much faster than looping properties in a random object.(http://jsperf.com/performance-of-array-vs-object/3)

Tips on v8 perf by Addy Osmani

There’s really only one major difference between objects and arrays in JavaScript, and that’s the arrays’ magic length property. If you’re keeping track of this property yourself, objects in V8 should be just as fast as arrays.

Array vs Objects is relevant?better traversal model?

So after all there is no difference and we can use a big nested array to mimic the fixed-length records in neo4j?

A Poc is done http://jsperf.com/javascript-graph/2

Testing in Chrome 30.0.1599.101 on OS X 10.7.4
Test Ops/sec
traverse random connected node by index
indexTranverse(large_random_nodes_offset,NODE_COUNT_LARGE);
17.19±1.20%79% slower
traverse random connected node by pointer
pointerTranverse(large_random_nodes, NODE_COUNT_LARGE);
22.95±41.98%80% slower
traverse ordered node by index
indexTranverse(large_ordered_nodes_offset,NODE_COUNT_LARGE);
62.43±83.32%fastest
traverse ordered node by reference
pointerTranverse(large_ordered_nodes, NODE_COUNT_LARGE);
103±26.84%fastest

To make result relevant, we are taking a large record – one million nodes.

Also we have cases where what each node connecting to is randomized.

The result is consistent that using pointer perform better in both ordered / random case in similar ratio, which benefit from reduced array lookup. And access in order really speed up – This implies the major benefit of optimizing is from localized memory access which help mechanisms like caching & reduce swapping bytes in memory.

Memory locality still matters

Thus, locality in array is still important consideration on where nodes should reside.

Some idea to investigate could be buckets of arrays of subgraph, periodic optimization to put connected nodes closer. Meanwhile, this should not limit the graph db to express itself that nodes being able to be freely connected to any node.

One place we can apply array directly is looping the relationships of a node.

just a start

I could be very wrong. This is just an idea to explore.

I don’t even have much experience in practical neo4j at the moment. so I will come back to this and see.

Discussions

There are quite a lot of related discussions / inspiring posts

http://stackoverflow.com/questions/614126/why-is-array-push-sometimes-faster-than-arrayn-value/614255#614255

http://stackoverflow.com/questions/7408734/building-an-object-graph-from-flat-object-array-in-javascript

http://mrale.ph/blog/2012/06/03/explaining-js-vms-in-js-inline-caches.html

So Mozilla’s Pancake is storing user browsing history & stuff in elastic search & neo4j. What if a Chrome extension or Firefox can do some of it directly. https://wiki.mozilla.org/PancakeInfrastructure#PancakeElasticSearchServer

A/D Test

With ripples in my work and my best friend introducing the book Clean Coder by Uncle Bob to me – bible of programmer as he put it.

Time to think again on professionalism of my work. Ethics is really something in my mind recently. I feel lost and empty that I need to think about it.

Before another story, lets put this quick thing that has been in my mind for a while. This is a WIP

There is a very famous Joel Test by Joel Sposky on how to code better.

I write this to give an idea how serious is the problem. Much lower level. It is very opinionated.

The term Application Developer, or equivalent like SA is very common in the industry.

If Turing Test tests whether there is a human or machine in the room, this A/D test tests whether one works as an application or as a developer. (or as System / as Analyst)

A/D Test

  • On average did you change more than 20 lines of code this week?
  • Did you ever heard of github?
  • So you use version control. You use or at least heard of someone in your office using a distributed version control systems?
  • Does projects related to your work missed 5 times for delivery date / end of testing cycles last month? The Actual complete work I mean.
  • Did you ever read anyone of these books: peopleware, mythical manmonth, clean coder
  • Do you have line or project managers that don’t code but micro-manage and judge your estimation on how to code
  • Besides the lang you using everyday, do you code in other languages after you graduate from college?
  • Did you watch 3 videos / blog posts / a books this month related to improving software engineering that is not a mandatory training from your work?
  • Do you spend less than 2 hours on meeting a day?
  • Do you automate test other than unit test?
  • Do you use shortcut maximize your screen / use ctrl R in bash to look for commands / ctrl + shift + f to reindent your code in eclipse / Alt shift R to change the name in scope ?
  • Are you not feeling a sense of surprise/disgust  when your fellow developer brings a mac / use its own IDE?
  • Do you have more monitors than your project manager? -1 for the reversed situation.

Very sad, if 3 is the passing mark I promise you 80% of the collegue in the big corp I am working in fail. I heard you say WTF. But it is really the fact

I don’t blame people that work like this. But we need a culture that either the title of Developer or the word professionalism should not be assoicated to their role. And more importantly, most of them should not manage me.

Convert HTML to ENML for Evernote – a non-trivial process

As title implies, this post is for Evernote hackers.

This function is technically one of the most crucial part for making Cheeatz, an editor to Evernote your Code with Gists and Markdown.

so my use case is: convert the HTML generated by Gist’s javascript and Markdown into ENML and save in Evernote.

(convertinng gist’s javascript and markdown into HTML is another non-trivial process, which is not scope of this article)

There are nice javascript libraries to work with Evernote, namely the official sdk. For manipulating ENML, I recommen enmljs by berryboy. This is a simple & handy util.

So enml.js has useful and well-named methods – enml.PlainTextOfENMLenml.HTMLOfENML, etc

the only thing missing is ENMLOfHTML(), which I need.

enmlOfHtml

Github: enmlOfHtml

html to enml, <a> to an <a><p> to a <p>, Sounds easy. Only later I found this is indeed a non-trivial process. I hacked it anyway and put it as above.

usage:

var enmlOfHtmljs = require('enmlOfHtml');

var html = '<html><p>put html here</p></html>';

//ENML is valid ENML that you can send to evernote for creation

enmlOfHtmljs.ENMLOfHTML(html,function(err,ENML){

    console.log(ENML);

});

You can go straight to try it, but a understanding is highly recommend:

Before go in to details, we need to understand the process of saving a note in Evernote.

I won’t cover those auth,token, etc where you can read in their documentation, but focus on ENML.

ENML

ENML is based on a subset of XHTML. There are rules and schema to follow, permitted and prohibited element which can be read here

What need to be done to convert HTML into ENML in Evernote server

From the documentation,

  1. Convert the document into valid XML
  2. Discard all tags that are not accepted by the ENML DTD
  3. Convert tags to the proper ENML equivalent (e.g. BODY becomes EN-NOTE)
  4. Validate against the ENML DTD
  5. Validate href and src values to be valid URLs and protocols

XML

As in step 1, the basic thing is you need to write XML.

here I used xml-writer which enmljs used

Dom or Not?

Some library write xml using tree-like structure or with DOM-likeapi. From my experience there is performance punishment to emulate the dom at node side (e.g. with jsdom). I choose to write those HTML straight

I have been trying with libxmljs, but I dont see advantage using it at the moment for building XML. However I believe for parsing this one is nice.

Since this use purely regex, this part should work in both client side and server side.

Dont escape those HTML!

One Caveat is you need to writeRaw to write characters, otherwise HTML will be escaped

Clean up your HTML

Then step 2 & 3 is the tricky part. Doing it with regex alone could be painful, but luckily I found this

module node-resanitize

I modifiy the library to support options on what attributes to escape.

also remember to replace body with en-note

CSS!

This is one of the most non-trivial part which is logically:

  1. there is link style sheet in HTML (as in gist)
  2. ENML dont support link tag.
  3. Luckily, style attribute is supported in most tags.

inline it!

=>so you need to extract that style sheet (download if needed) and inline it as attribute.

luckily, there is a bigger audience for this problem. Another place posing similar requirements are what you use day to days, Email.

So there are some good libraries out there. Styliner is excellent.

Meanwhile, it used Q and result is returned inside the callback, and this make this enmlOfHtml put result into callback as well.

Note the 5th step – values in href and src must be valid URLs and protocols

This is what I missed and somehow created a bug.

At the time of writing, github changed their javascript to render one of the link without the gist domain –>actually a bug

so instead of href="https://gist.github.com/vincent....", there is href="/vincent..."

Then when user try to create Note in my site, it fails as when I call the create Note api there is an error

{ errorCode: 11, 

parameter: 'Error processing document: Invalid a href attribute:vincentlaucy/5548010/raw/29e88cc4f84422df5febadf93b10227f4c894c9b/gistfile1.js' } 

With try and error, to get Evernote accept your ENML, it must start with :// at least

Some Similar implementation is in Sanitize, where you can pass options on what to accept (e.g. ftp://, http:// etc), just it is client side.

These values should be either removed / replace with default / current domain to pass the validation.

I put a simple regex for that purpose.

Side-track: this is why you should always write “learning test” against external api

Make it better: Local Validation

I didnt mention step 4- validation

As metnioned in Evernote’s Docuemntation

Note: While it is possible to rely on the Evernote Cloud API to validate the ENML of your notes, we recommend downloading the DTD file (linked above) and use it to validate your note’s XML within your app. A few reasons this is a good idea:

  • Note validation will be much faster when performed locally.
  • Note validation can be performed offline.
  • The results of validating your notes locally will be the same as if you were to rely on the Evernote Cloud API to validate your ENML.

So Evernote is using DTD but not XSD, I googled a little bit on using node for DTD validation, however seems no javascript library available at the moment. Let me know if you found one.

Make it better: more

so I put a trivial implementation for this non-trivial process, but more worth to be done

  • test casessss
  • make this module support requirejs
  • it on client side
  • find/create a module that is good at both client side and server side HTML sanitize, with generic options

Hope you find it useful.

Happy to tell you this blog post produced using Cheeatz

Introducing Cheeatz: Evernote your code with Gists and Markdown

Last 2 months I have been working hard on Cheeatz

By Cheeatz – that is cheat sheet. (and I hope it sounds like Cheese for you)

It is on Evernote Dev Cup

Basically right now it is what the pitch line says:

Evernote your code with Gist and Markdown

so with Cheeatz you can write in markdown, embed your gist and save it into evernote – with all those formatting.

Demo

Write markdown and save gist into evernote.

For HOW-TOs, [visit the site](cheeatz.com/editor) or watch this video.
It is good for both non-developers and developers.

Basically with this web app:

  • you input your markdown with the live-preview,
  • If you are developer and writing about some code – embed gist by id when you want!
  • save to Evernote and open it for modify later
  • and share it to your friends.

Use Cases

  • you are creating cheatsheet
  • you are creating documentation
  • you are jotting notes against some code

Upcoming / Missing features:

  • create gist right away from the editor
  • Tagging
  • Code language-syntax highlighting in editor

Why Evernote

I am addicted to Evernote and I think it provides really nice features – esp synchronization and search, of course.

There are various ways to embed code in a blog post or documentations. Gist is great gaining popularity. Nice syntax highlighting and one thing definitely important – every gist is a git repo. The only thing missing is pull request, and Cheeatz is little bit into that.

Also for sure the Evernote DevCup is very kind to developers – thus I started to create this prototype on Evernote’s API

It is non-trivial

you may first think why don’t I just clip the gist and paste into evernote.

It work in some scenarios, but what If your gist changes? your markdown changes? You definitely needsynchronization.

Also most blogging platforms allow you to embed gist and show it in browser, but we also render the gist view and save it to your Evernote.

You can view in Evernote your gist with all those highlighting and search it, even offline.

But Why

This is a prototype, what I aim is something bigger.

The current prototype missed the most important feature: search for Cheeatz and collaborate for it.

There has been too many times that when I code I google for quick snippets. I look for stuff likejava buffered input stream or unix kill process by name but I need to read through many pages to get that simple code.

A recent example, I was working with a Java expert and I saw him literally google for Thread.sleep()example and paste it into the code. Whay he got is the Thread.sleep with the try catch.

Admit it, We all cheat

Thanks to google I can be a programmer with my bad memory. “Cheating” for a solution provides you the correct syntax, quick answer and more importantly, best practices insights on stuff that you never thought of.

Cheating Ecosystem

What we need is a platform that we can cheat better, together.

Again, gist is great. But there too many gists. Trust the cloud, voting up and down will be helpful.

Also we will need ways that people can contribute to the same cheat sheet. What if every cheeatz is a git repository as well? then you got all those familiar clone-pull requests.

With Markdown you write less

I only discovered markdown this year but it is definitely game-changing.

I have been using different approaches to sync markdown and evernote, but for seamless integration I will need to create my own.

Stackoverflow is there

Stackoverflow is great. It literally saves days and years for developers. You know what I mean.

However, sometimes I feel that Stackoverflow should be places for problems that required detailed and quality discussion. As its policy says, google and read the documentation before you ask the question. For questions like how to loop an array in java or how to wait the document ready in jQuery, that should really be other places.

Last Year I was learning Groovy at company and really it is Groovy Goodness by mrhaki that taught me how to. Sometimes you know what you want to do – loop an array or so, just you need to get damn syntax right. or sometimes you don’t know what you are doing – you see those examples and you got some idea. That site is excellent – you google it and it provides examples and expected results in a precise manner. Many cases you just copy, paste and modify a little bit you get what you want. This is quite inspiring for me.

Save some brain power

Personally I am little bit into cognitive science and thus I believe providing the most efficient information retrieval is really something important. Unified format. Applicable platforms. Executable code readily result will ease our brain Time are saved and less neurons are distracted.

A little bit more serious than my stuff before, at least I bought my very first domain name.

Stack

At the moment, it is Node.js/express.js/require.js/Redis. Yeoman for dev. Trying to put AngularJs into the picture.

Thanks to many more node libraries, I will write a little bit more on this latter.

So at the very least this blog post is written in the editor of Cheeatz I created, and I think it helped.

Thanks to my friends @gilbertwat and @westerpantz for contributing and make this happening.

Now I need to rush for my other projects – but let me know your thoughts on this. I do agree it might be quite fragile and slow right now, but if I can learn that it is something really worth the effort I will definitely improve it, and I believe this is how everything started.