High semantic similarity between xkcd comic and thesis
September 23, 2011
It’s a wrap!
April 13, 2012
This category can soon be archived ;)! Earlier this week I handed in my final paper, and yesterday was the day of my final presentation. It was a great day and I’m really excited about embarking on my next adventure. I will soon start as a PhD candidate at the University of Amsterdam, on a very […]
text graphs
April 12, 2012
Measure and Visualize Semantic Similarity Between Subgraphs
March 19, 2012
As I blogged previously, I am working on measuring the performance of my keyword extraction algorithms. The confusion matrix approach I have implemented is quite ‘harsh’. It ignores any semantic information and simply treats the concepts as words, and counts hits and misses between two sets of concepts. To benefit from the semantic information described […]
Abstract
March 10, 2012
Below the first draft of the abstract of my paper. It doesn’t yet include the results/conclusion. Word count: 127 Semantic annotation uses human knowledge formalized in ontologies to enrich texts, by providing structured and machine-understandable information of its content. This paper proposes an approach for automatically annotating texts of the Cyttron Scientific Image Database, using the […]
Algorithm performance measurement: Confusion Matrix
March 3, 2012
As I am starting to gather testing data, I figured it’d be a good time to determine how to measure the performance of the different results of my 24 text representation algorithms. What I want to measure per algorithm: how many keywords are predicted the same as the experts, and how many aren’t. After some research […]
Force-Directed Graphs: Playing around with D3.js
February 27, 2012
Update: Newer example of Force-Directed d3.js Graph here: Measure and Visualize Semantic Similarity Between Subgraphs I recently replaced python-graph in my code with NetworkX, a slightly more sophisticated graph library for Python. Besides some more advanced algorithms for graph analysis (comparison, unison etc.) which can prove useful when analyzing data (comparing human data with mine, for […]
Not dead (yet)
February 10, 2012
While I haven’t been as active and hard working on my graduation project as I would have liked to be, I am not dead (nor the project). Earlier this week I presented my project to the Bio-imaging group of Leiden University, which helped me a lot. I was able to present my project pretty much […]
Pathfinder finds paths!
November 20, 2011
First results are in! My computer spent about 20 hours to retrieve and store neighboring concepts of over 10.000 concepts which my Breadth-First-Search algorithm passed through to find the shortest paths between 6 nodes. But here is the result, first the ‘before’ graph, which I showed earlier: all retrieved concepts with their parent relations. Below that is […]
Ontology-based Text Visualization: Towards a Visual Language
November 12, 2011
The first steps towards creating graph-based text visualizations is thinking about how I would like to visually represent the structure of the information I extract from texts. I have included a preliminary example of the output at the bottom of this page. I use three different methods of extracting concepts from a piece of text: […]
Add some #dataviz in the mix
October 25, 2011
All this time I have been working on my ontology-powered topic identification system: combining semantic web technologies with natural language processing and text-mining techniques to extract (a) topic(s) from a text. But I never really decided what my program should output: a topic? A list of potential topics with their ‘likeliness’? That would mean the […]
Computing string similarity with TF-IDF and Python
October 3, 2011
“The tf–idf weight (term frequency–inverse document frequency) is a weight often used in information retrieval and text mining. This weight is a statistical measure used to evaluate how important a word is to a document in a collection or corpus.”[wikipedia] It is also the weight I use to measure similarity between texts, for these two […]
My thesis in pictures
September 20, 2011
Because sometimes six pictures are worth a trillion triples.
Python graphs and visualizations
September 18, 2011
To my right is a visualization of the output of my SPARQL-powered shortest path algorithm, finding a link between ‘intracellular and extracellular accumulation’ & ‘developmental and adult structural defect’, 2 concepts in the Mouse Pathology ontology. Click it! It shows the two ‘source’ concepts in white, and the shortest path (of 3 nodes: 4 hops) in […]
Simple keyword extraction in Python: choices, choices.
September 12, 2011
As explained in an earlier post, I am working on a simple method of extracting ‘important words’ from a text-entry. The methods I am using at the moment are frequency distributions and word collocations. I’ve bumped into some issues regarding finetuning my methods. Read on for a short explanation of my approaches, and some issues […]

Thesis (MSc)

High semantic similarity between xkcd comic and thesis

High semantic similarity between xkcd comic and thesis

Related

Leave a ReplyCancel reply