{"id":21627,"date":"2018-12-06T15:40:07","date_gmt":"2018-12-06T23:40:07","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=21627"},"modified":"2018-12-06T15:41:13","modified_gmt":"2018-12-06T23:41:13","slug":"better-faster-graph-processing","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/","title":{"rendered":"Better, Faster Graph Processing"},"content":{"rendered":"<p><em>MIT CSAIL&#8217;s new programming language GraphIt outperforms other models 5x while reducing lines of code<\/em><\/p>\n<p>Every time you Google sometime, you\u2019re benefiting from something called graph data. Google represents everything on the web as a giant graph of different webpages and their relationships to each other. (This is in contrast to representing data as essentially a series of rows and columns.)<\/p>\n<p>So-called \u201cgraph processing\u201d is used for all sorts of things in our lives, from Uber driving directions to Facebook \u201cPeople You May Know\u201d friend recommendations. However, there have been many limits in terms of being able to use graph processing on a larger scale. As a result, programmers often have to use different combinations of techniques that make trade-offs in terms of efficiency and flexibility.<\/p>\n<p>However, a team from MIT CSAIL has developed a new programming language for graph processing that could help. Dubbed \u201c<a href=\"http:\/\/graphit-lang.org\/\" target=\"_blank\" rel=\"noopener\">GraphIt<\/a>,\u201d the new domain-specific language has been shown to outperform existing frameworks by a factor of nearly 5x while also reducing the lines of code by almost an entire order of magnitude.<\/p>\n<p><iframe loading=\"lazy\"  id=\"_ytid_79599\"  width=\"480\" height=\"270\"  data-origwidth=\"480\" data-origheight=\"270\" src=\"https:\/\/www.youtube.com\/embed\/ptIVf-YlkhY?enablejsapi=1&#038;autoplay=0&#038;cc_load_policy=0&#038;cc_lang_pref=&#038;iv_load_policy=1&#038;loop=0&#038;modestbranding=0&#038;rel=1&#038;fs=1&#038;playsinline=0&#038;autohide=2&#038;theme=dark&#038;color=red&#038;controls=1&#038;\" class=\"__youtube_prefs__  epyt-is-override  no-lazyload\" title=\"YouTube player\"  allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen data-no-lazy=\"1\" data-skipgform_ajax_framebjll=\"\"><\/iframe><\/p>\n<p>GraphIt generates fast implementations for algorithms with different performance characteristics running on graphs with different sizes and structures. According to MIT CSAIL PhD student Yunming Zhang, it separates what is computed from how it is computed. Programmers specify the algorithm using an algorithm language, and performance optimizations are specified using a separate scheduling language. The algorithm language simplifies expressing the algorithms, while exposing opportunities for optimizations.<\/p>\n<blockquote><p>GraphIt enables programmers to easily search through this complicated tradeoff space by composing together a large set of edge traversal, vertex data layout, and program structure optimizations,\u201d says Zhang, first author on a paper about GraphIt that was presented this month at the annual conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA) in Boston. \u201cThe separation of algorithm and schedule also enables us to build an autotuner on top of GraphIt to automatically find high-performance schedules.\u201d<\/p><\/blockquote>\n<p>Zhang co-wrote the paper, &#8220;<a href=\"https:\/\/arxiv.org\/pdf\/1805.00923.pdf\" target=\"_blank\" rel=\"noopener\">GraphIt: A High-Performance Graph DSL<\/a>,&#8221; with MIT professors Saman Amarasinghe and Julian Shun, alongside MIT undergraduate Mengjiao Yang, MIT CSAIL postdoc Riyadh Baghdadi and Shoaib Kamil of Adobe Research.<\/p>\n<p>&nbsp;<\/p>\n<p><em>Sign up for the free insideBIGDATA\u00a0<a href=\"http:\/\/insidebigdata.com\/newsletter\/\" target=\"_blank\" rel=\"noopener\">newsletter<\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A team from MIT CSAIL has developed a new programming language for graph processing that could help. Dubbed \u201cGraphIt,\u201d the new domain-specific language has been shown to outperform existing frameworks by a factor of nearly 5x while also reducing the lines of code by almost an entire order of magnitude.<\/p>\n","protected":false},"author":10513,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[87,180,56,84,1,85],"tags":[714,96],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Better, Faster Graph Processing - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Better, Faster Graph Processing - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"A team from MIT CSAIL has developed a new programming language for graph processing that could help. Dubbed \u201cGraphIt,\u201d the new domain-specific language has been shown to outperform existing frameworks by a factor of nearly 5x while also reducing the lines of code by almost an entire order of magnitude.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2018-12-06T23:40:07+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-12-06T23:41:13+00:00\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/\",\"url\":\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/\",\"name\":\"Better, Faster Graph Processing - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2018-12-06T23:40:07+00:00\",\"dateModified\":\"2018-12-06T23:41:13+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Better, Faster Graph Processing\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning &amp; Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Better, Faster Graph Processing - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/","og_locale":"en_US","og_type":"article","og_title":"Better, Faster Graph Processing - insideBIGDATA","og_description":"A team from MIT CSAIL has developed a new programming language for graph processing that could help. Dubbed \u201cGraphIt,\u201d the new domain-specific language has been shown to outperform existing frameworks by a factor of nearly 5x while also reducing the lines of code by almost an entire order of magnitude.","og_url":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2018-12-06T23:40:07+00:00","article_modified_time":"2018-12-06T23:41:13+00:00","author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/","url":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/","name":"Better, Faster Graph Processing - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2018-12-06T23:40:07+00:00","dateModified":"2018-12-06T23:41:13+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2018\/12\/06\/better-faster-graph-processing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Better, Faster Graph Processing"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning &amp; Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-5CP","jetpack-related-posts":[{"id":29025,"url":"https:\/\/insidebigdata.com\/2022\/04\/13\/accelerating-graph-technology-adoption\/","url_meta":{"origin":21627,"position":0},"title":"Accelerating Graph Technology Adoption","date":"April 13, 2022","format":false,"excerpt":"In this contributed article, Keshav Pingali, CEO and co-founder of Katana Graph, believes that graph technology\u2019s need is apparent \u2013\u00a0and will be even more so in the coming years with 95% of businesses stating that managing unstructured data is a serious challenge. Highlighting successes and promoting interoperability with other libraries\u2026","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":9174,"url":"https:\/\/insidebigdata.com\/2014\/05\/11\/call-participation-graph500-green-graph500-lists\/","url_meta":{"origin":21627,"position":1},"title":"Call for Participation: The Graph500 and Green Graph500 Lists!","date":"May 11, 2014","format":false,"excerpt":"\"The importance of Big Data processing is steadily growing. Numerous large-scale problems that represent social, physical, or financial interactions are most naturally modeled with graph representations. Such graphs requiring the processing and storage of billions of vertices and graph edges, high-performance graph computations to process them efficiently. The Graph500 list\u2026","rel":"","context":"In &quot;Featured&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2014\/05\/graph500-logo.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1940,"url":"https:\/\/insidebigdata.com\/2012\/09\/26\/yarcdata-shows-why-big-data-is-more-than-hadoop\/","url_meta":{"origin":21627,"position":2},"title":"YarcData Shows Why Big Data is More than Hadoop","date":"September 26, 2012","format":false,"excerpt":"Gartner's Carl Claunch writes that Big Data involves a much wider and varied set of needs, practices, and technologies than just Hadoop or data warehouses. To meet the needs of graph computing, Cray's YarcData systems feature a single systemwide memory space that does not need to be partitioned. YarcData has\u2026","rel":"","context":"In &quot;Big Data Software&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25058,"url":"https:\/\/insidebigdata.com\/2020\/09\/30\/tigergraph-unveils-free-tigergraph-enterprise-edition-helping-companies-use-graph-as-the-foundation-of-many-modern-data-analytics-and-ai-capabilities\/","url_meta":{"origin":21627,"position":3},"title":"TigerGraph Unveils Free TigerGraph Enterprise Edition, Helping Companies Use Graph as the Foundation of Many Modern Data, Analytics and AI Capabilities","date":"September 30, 2020","format":false,"excerpt":"TigerGraph, the scalable graph database for the enterprise, announced free licenses for TigerGraph Enterprise, an offering that will empower customers to easily use graph analytics and algorithms that quickly traverse graphs to find insights in real-time. TigerGraph is now free for everyone to use for databases up to 50GB graph\u2026","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":15048,"url":"https:\/\/insidebigdata.com\/2016\/05\/19\/tibco-graph-database-and-project-flogo-bring-smarter-connectivity-and-real-time-insights-to-iot-and-digital-business\/","url_meta":{"origin":21627,"position":4},"title":"TIBCO Graph Database and Project Flogo Bring Smarter Connectivity and Real-Time Insights to IoT and Digital Business","date":"May 19, 2016","format":false,"excerpt":"TIBCO Software Inc., a global leader in integration and analytics, announced Project Flogo\u2122, an ultra-lightweight integration software solution, and TIBCO\u00ae Graph Database, a translytical database for big data.","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":17357,"url":"https:\/\/insidebigdata.com\/2017\/03\/10\/graph-databases-connected-enterprise\/","url_meta":{"origin":21627,"position":5},"title":"Graph Databases and the Connected Enterprise","date":"March 10, 2017","format":false,"excerpt":"In this special guest feature, Emil Eifrem, Founder and CEO of Neo Technology suggests that in order to achieve connected enterprise status and realize the significance of the graph database, companies must understand the database options that are available. Right now, the landscape consists of three categories, which he outlines\u2026","rel":"","context":"In &quot;Data Storage&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/21627"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=21627"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/21627\/revisions"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=21627"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=21627"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=21627"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}