{"id":32223,"date":"2023-04-28T07:00:00","date_gmt":"2023-04-28T14:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=32223"},"modified":"2023-04-27T10:34:11","modified_gmt":"2023-04-27T17:34:11","slug":"if-big-data-is-the-immovable-object-enterprise-search-is-the-unstoppable-force","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/04\/28\/if-big-data-is-the-immovable-object-enterprise-search-is-the-unstoppable-force\/","title":{"rendered":"If Big Data Is the Immovable Object, Enterprise Search Is the Unstoppable Force"},"content":{"rendered":"
\n
\"\"<\/figure><\/div>\n\n\n

In this special guest feature, Elizabeth, Director of Sales at dtSearch Corp.<\/a>, asks ‘How do you approach Big Data?’ You could try to organize the heck out of it if you have all of the time in the world and your data isn\u2019t constantly changing. Or you could kick back and let enterprise search provide immediate access. If Big Data is the immoveable object, enterprise search is the unstoppable force. dtSearch offers enterprise and developer products running “on premises” or in the cloud to instantly search terabytes with over 25 search options. dtSearch\u2019s own document filters support files, emails, databases and web data.<\/em><\/p>\n\n\n\n

How do you approach Big Data? You could try to organize the heck out of it if you have all of the time in the world and your data isn\u2019t constantly changing. Or you could kick back and let enterprise search provide immediate access. If Big Data is the immoveable object, enterprise search is the unstoppable force.<\/p>\n\n\n\n

Whereas a scan-the-Internet search engine like Google crawls the web, enterprise search lets you do an in-depth exploration of your own Big Data. To instantly search terabytes, enterprise search first has to index the data. The index is simply an internal guide that pre-tabulates unique words and numbers in the data and the specific location of each, including across multiple data repositories and locations.  <\/p>\n\n\n\n

Indexing, a technical overview.<\/strong> Let\u2019s start with what the indexer does *not* do. It does *not* move, copy, delete or in any way alter original files. And it does *not* pull up files in their associated applications \u2013 like you would review a Microsoft Word document in Word or a PDF in Adobe Acrobat Reader. Such an approach would take way too long. <\/p>\n\n\n\n

So, what *does* the indexer do? The indexer goes straight to the binary format of all files. If you looked at a binary format, you\u2019d see a mess of binary codes, making it hard to read individual words much less complete sentences. However, a search engine can tackle binary formats because it has built-in document filters.<\/p>\n\n\n\n

The document filters need to apply the correct parsing specification to each binary format before indexing. Different file types, and sometimes even different versions of the same file type, will have their own custom parsing specifications, some hundreds of pages long. Without the right parsing specification, parsing the text of a binary format will quickly hit a dead end. <\/p>\n\n\n\n

The indexer, unleashed. <\/strong>With all this emphasis on precision parsing, you might expect indexing to take a lot of effort. While the document filters have their work cut out for them in the data recognition department, all you have to do is point to the folders, email repositories, etc. to cover, and let the indexer do the rest. On its own, the indexer can figure out the parsing specification to apply to each binary format. (A search engine needs to review the binary format for this determination, not the file extension. Saving a PDF with a .DOCX extension or an Access database with a .ONE extension is all too easy.)<\/p>\n\n\n\n

On the plus side, the indexer can review the data on a much deeper level than a human looking at files in their associated applications. For example: <\/p>\n\n\n\n