Sorry for the massive PR, but it was difficult to break this up into small pieces.

Changes

• Refactored each plugin into it's own module (no longer modifies globals)
• Added test and build scripts (use npm test and npm run build respectively)
• Added Node.js support while maintaining browser support (using browserify)
  • Running npm run build writes the browser bundle to ./dist/knwl.js
• Running tests no longer requires globally installed npm packages
• Plugins are now registered explicitly using knwl.register(name, definition)
  • All default plugins are automatically registered for you
• Renamed knwl.parserList to knwl.plugins

Todo

• Update docs to reflect these changes.
• Make sure everything in the demo still works as expected.

Let me know if you have any issues with what I've done here. I'm happy to update this PR.

You will need to enable travis-ci and put the status image in your self.

-- edited out. tests now pass -- I will add more tests as I go along but this seems like a good place to start off with.

Parsed mozilla’s list of tld names to recognize links written without http:// or www. http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat

This should solve #33 Apparently the JS-file is now crowded. :(

This is a major source update Pinging @loadfive @gprasant for feedback.

While working on knwl.js I noticed a great deal of the tabbing was all over the place. I have gone though and enforced everything is 4 spaces for a tab. This means alot of the code is now much more readable on how "deep" you are.

I also replaced all uses of alert with console.error so that if/when we move to a node module or so mething that could be compatible with node. It will be a lot smoother.

I also made sure that all the multidimensional array's are spaced out more, making appending to them a great deal more easy.

Overall I think this will greatly improve the code readability of the whole project!

Have a great xmas :christmas_tree:

as a major side note, it is worth merging this one last before anyone else's PR's to avoid death by merge conflicts

Now after cloning the project, they only need to run npm install -g from the base project directory to globally and recursively install all required node modules (keep in mind if someone just runs npm install the node_modules folder will appear in the base project directory. Should consider adding a .gitignore if you do not want the node_modules folder appearing in the project).

This will create an easier way for all contributors to manage/install dependencies for the project.

Hey,

I was just looking through the source code - Nice work! I just noticed a really small JS tweak you could do that would make it a little tidier. At the moment you do this a few times:

var words = []; //make a copy of the rawWords array (otherwise, changes will be mirrored to knwl.words prop)
        for (var i = 0; i < rawWords.length; i++) {
            words[i] = rawWords[i];
        }

When you could do either: var words = rawWords.concat([]); or var words = rawWords.slice(0);

As I say, it really doesn't matter too much... but clean code is happy code :smile: and I doubt the performance impact (if any) is significant.

Presently all functionality stems from data stored on the rootmost require('knwl') object. This makes it impossible to have more than one parse running at a time. I would like to see the ability to instantiate individual parser instances each parsing different strings.

It appears that all the tests were removed in 7fd3637, was there a reason behind this? It would be much easier to write plugins if we had example test cases for the existing default ones.

The code should be quite clean. Let me know if there are any issues. This is currently an experimental plugin because:

• 12:30 is usually not a ratio but some time of the day. Yet still it gets recognized as a ratio.
• context is not tested and unstable.

I will implement a matcher for financial values in the following days. Not sure if a new release makes sense before both plugins are merged.

Using the knwl.js for analysis some links are very good too! Maybe detect the type of the link like, article, video, picture and other stuffs are a good improve.

I know it's early days, but it'd be cool to start it now, to make things easier in the future. Being able to set what language the text is using, and then running any of the plugins with that specific language will make for more detailed analysis of processed text.

Hey there!

I'd like to report a security issue but cannot find contact instructions on your repository.

If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.

Thank you for your consideration, and I look forward to hearing from you!

(cc @huntr-helper)

Hello,

The GitHub Security Lab team has found a potential vulnerability in your project. Please create a Security Advisory and invite me in to further disclose and discuss the vulnerability details and potential fix. Alternatively, please add a Security Policy containing a security email address to send the details to.

Kind regards, A

I am trying to detect a phone number as a Number and it seems that Knwl.js accepts only strings and the following TypeError occurs.

TypeError: str.toLowerCase is not a function
    at Knwl.init (/path/knwl.js:1154:23)

Here is a test case that reproduces the problem.

var knwl = new Knwl();
var phone = 5555555555;
knwl.init(phone);
console.log(knwl.get('phones'));

To fix the problem, we may change line 1154 to the following

var lowercase = String(str).toLowerCase();

Hello, everyone. First off, I would like to express my gratitude to all who have contributed to Knwl.js; this project wouldn't be the same without you. Looking ahead, however, a few things need to be addressed.

I wrote the base framework for this project many years ago, and I did so while also in the midst of becoming comfortable with Javascript, and, as such, the initial design of the program was sporadic; it was more of an experiment in personal interests than a project with a definitive usecase. Since then, and through many refinements and evolutions, Knwl.js has developed into a program that functions decently on a restricted number of datapoints. For certain use cases, these limitations are not an issue; however, in more cases than not, the inflexible methods by which Knwl.js functions inhibit the potential uses of the program.

Knwl.js works by referencing a limited dictionary of terms and combinations of terms, which, when found in a body of text, trigger certain methods to run a hard-coded, systematic scan through the text for related information. This static, unchanging nature lends itself to Knwl.js' inability to adapt and evolve independently. In order to combat this issue, a new version of Knwl.js was developed to support plugins. The update allowed developers to create parsers for specialized tasks as needed and somewhat mitigated the rigidity of Knwl.js. Unfortunately, when a foundation is faulty, anything built atop will achieve nothing more than instability, and I am afraid this is the case with Knwl.js.

The framework that Knwl.js has been built upon serves its purpose adequately; the project was built to be a simple, plug and play parser built in javascript; however, for the reasons stated prior, I do not believe there is potential for further growth in its current implementation. For this reason, I see little reason to continue attempting without facing the original problem: a static, pre-defined dictionary-dependant parser.

Knwl.js has a special place in my heart as a project I conceived very early on. If you use this project, do not worry: it will not be going away; however, though I will monitor pull-requests and merge as the community suggests, I will no longer be actively writing for the project.

In a word, I am giving this project back to you, the people who made it so great in the first place. I am open to any suggestions or comments you may have. I would like to do what is best for all of you.

Thank you, Ben

Hi @loadfive,

Thanks for this cool tool, I'm using it to parse some dates in English and that's quite good :)

However I want to help you to increase the translation range but the structure of the project doesn't seem to be adequate to the extensibility required for contributions.

As I see, we have a lot of translatable strings mixed with the functionality... It would be so much better to separate functionality/feature from the string searched/used. Basically making a small dictionary for the functions.

I saw you have - on the experimental_plugins - an english.js which contains some words already.

That'd be nice if you hadn't mixed some functionality (like 'findSentences' and alike).

Last but not least, consider doing a language identification based on the culture, not on the language itself. E.g:

pt_PT !== pt_BR

en_US !== en_UK

and so on.

If you want I can help out, just tell me if you'd like some help on that ;) And thanks for your time!

One thing that would be interesting to check is the possibility to integrate either with a tool/library like FormatJS or with the Unicode Group, more exactly the ICU sub-group ( http://site.icu-project.org/ ).