{"id":1534,"date":"2018-11-22T15:01:04","date_gmt":"2018-11-22T13:01:04","guid":{"rendered":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/?p=1534"},"modified":"2023-08-11T09:39:00","modified_gmt":"2023-08-11T07:39:00","slug":"python-applied-to-machine-learning-and-data","status":"publish","type":"post","link":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/blog\/bi-data-analytics\/python-applied-to-machine-learning-and-data\/","title":{"rendered":"Python Applied to Machine Learning and Data"},"content":{"rendered":"<p><em>An overview by Travis E. Oliphant.<\/em><\/p>\n<p>Most tech companies today are struggling to figure out how they can best work with their data. Travis E. Oliphant, the founder of startups <a href=\"https:\/\/www.quansight.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Quansight<\/a> and <a href=\"https:\/\/www.anaconda.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Anaconda<\/a>, and the primary developer of NumPy and SciPy packages for Python, gave a talk at North Star AI conference, powered by <a href=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/about-proekspert\/\">Proekspert<\/a> on how Python can be used effectively for machine learning and data \u2013 the heart of ML-driven technology.<\/p>\n<p>Oliphant\u2019s connection to Python dates as far back as 1997, when he was working with version 1.4. The very first problem he focused on was actually a data problem, and this led him to understand that in order to do anything with machine learning you have to get the data right.<\/p>\n<p>While data seems to be everywhere these days, said Oliphant, the biggest problem is how you gain access to and utilize that information.<\/p>\n<h3>Applying Python to machine learning problems<\/h3>\n<p>In Oliphant\u2019s mind, Python is one of the best languages to apply to machine learning problems. During his talk, he gave a thorough overview as to why and presented how he imagines artificial intelligence might develop in the future.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-1536 lazyload\" data-src=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant1.jpg\" alt=\"\" width=\"1200\" height=\"630\" data-srcset=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant1.jpg 1200w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant1-300x158.jpg 300w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant1-1024x538.jpg 1024w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant1-768x403.jpg 768w\" data-sizes=\"(max-width: 1200px) 100vw, 1200px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1200px; --smush-placeholder-aspect-ratio: 1200\/630;\" \/><\/p>\n<h2>Not Artificial but Augmented Intelligence<\/h2>\n<p>When Oliphant thinks of AI, he thinks of \u201caugmented\u201d rather than \u201cartificial intelligence.\u201d For at least the next fifty years this technology will be more about \u201cempowering people rather than replacing them,\u201d he said. To be sure, AI might take over some tasks that you are doing today, but then you will shift to doing something more important, while the machine takes care of more mundane assignments.<\/p>\n<p>What can AI be used for? Oliphant noted a multitude of possibilities.<\/p>\n<p>Any time you have a complex function with many variables for which you want to have an understandable input and output\u2014you can apply AI. For example, self-driving cars, medicine, and geophysics are a few of the fields where AI technology can make a big difference.<\/p>\n<h4>Big companies using Python in machine learning<\/h4>\n<p>In order to apply ML and be successful with your applications, Oliphant noted, you will need to work with people who have domain expertise, who know the business. The good news is that every major tech company is now involved in the AI field. Microsoft, Google, Apple, IBM, and Amazon are pioneering machine learning and artificial intelligence research.<\/p>\n<ul>\n<li>Google is developing its <a href=\"https:\/\/ai.google\/research\/teams\/perception\/\" target=\"_blank\" rel=\"noopener noreferrer\">Perception APIs<\/a><\/li>\n<li>Microsoft has its <a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/cognitive-services\/\" target=\"_blank\" rel=\"noopener noreferrer\">Cognitive Services for Azure<\/a><\/li>\n<li>IBM is continually building <a href=\"https:\/\/www.ibm.com\/watson\/\" target=\"_blank\" rel=\"noopener noreferrer\">Watson<\/a><\/li>\n<li>Amazon Web Services is also launching <a href=\"https:\/\/aws.amazon.com\/machine-learning\/\" target=\"_blank\" rel=\"noopener noreferrer\">AI Services<\/a><\/li>\n<li>Apple is developing <a href=\"https:\/\/developer.apple.com\/machine-learning\/\" target=\"_blank\" rel=\"noopener noreferrer\">Core ML<\/a><\/li>\n<\/ul>\n<p>These AI applications, said Oliphant, are on the verge of broad usability. Moreover, they are all written in Python.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-1535 lazyload\" data-src=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant2.jpg\" alt=\"\" width=\"1200\" height=\"630\" data-srcset=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant2.jpg 1200w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant2-300x158.jpg 300w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant2-1024x538.jpg 1024w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant2-768x403.jpg 768w\" data-sizes=\"(max-width: 1200px) 100vw, 1200px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1200px; --smush-placeholder-aspect-ratio: 1200\/630;\" \/><\/p>\n<h3>Obstacles in the Industry<\/h3>\n<p>There is amazing promise in the AI and machine learning industry, but we have a very long way to go, said Oliphant. The current landscape contains challenges such as organizational infrastructures that make data-sharing difficult, and out-of-date regulatory structures that were created for a different era. Technology is changing faster than education can keep up, and software is lagging behind hardware advances; programmers are not yet tapping into the full potential of the hardware that is available to them.<\/p>\n<p>As quickly as things progress, there remain many silos of technological advancement and a general lack of integration when it comes to methodology. \u201cCan\u2019t we figure out frameworks that everyone can use?\u201d asked Oliphant.<\/p>\n<p>AI exists in some basic forms now, but the dream is something bigger. There is still so much that needs to be done in order for the promise of AI to become actual capability, said Oliphant.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-1537 lazyload\" data-src=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant3.jpg\" alt=\"\" width=\"1200\" height=\"630\" data-srcset=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant3.jpg 1200w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant3-300x158.jpg 300w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant3-1024x538.jpg 1024w, https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-content\/uploads\/2018\/11\/Oliphant3-768x403.jpg 768w\" data-sizes=\"(max-width: 1200px) 100vw, 1200px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1200px; --smush-placeholder-aspect-ratio: 1200\/630;\" \/><\/p>\n<h3>Anaconda, a Possible Solution<\/h3>\n<p>Launched by Oliphant himself, Anaconda is an open-source tool that simplifies package management and deployment in the Python and R programming languages. It can be used to great effect for data science and machine learning applications. Moreover, its open-source package and environment manager, Conda, is language-agnostic and can distribute software for any language.<\/p>\n<h4>Ecosystem frameworks<\/h4>\n<p>When it comes to AI, Python is not enough, explained Oliphant. You need ecosystem frameworks to solve your problems as well as machine learning tools\u2014and Anaconda can bring all of these technologies together. \u201cOne of the key things we need is AI integrators bringing people these capabilities in everyday applications.\u201d<\/p>\n<p>Everybody loves modeling, predicting, classifying, and visualizing in AI, and these are fairly easy tasks to complete. The harder things are feature labeling, data-cleaning, data-extractions, deploying, reproducing, and scaling\u2014and this is where Anaconda can help.<\/p>\n<p>Oliphant discussed two other Anaconda tools, Numba and Dask, that he believes are crucial for anyone working in the realm of machine learning. <a href=\"https:\/\/numba.pydata.org\/d\" target=\"_blank\" rel=\"noopener noreferrer\">Numba<\/a>, he explained, is designed to help with scaling up. It is an open-source Python compiler that comes with a CUDA simulator. It can also compile for the CPU and GPU at the same time and make array processing easy. Most importantly, Numba executes code ~2.7 times faster than NumPy.<\/p>\n<p><a href=\"http:\/\/dask.pydata.org\/en\/latest\/\" target=\"_blank\" rel=\"noopener noreferrer\">Dask<\/a> is a parallel computation library for scaling NumPy arrays and Pandas dataframes. With Dask you can make a collection of arrays or dataframes that are larger-than-memory and can be used in distributed environments. It has a task scheduler that is optimized for computation, helping to run your custom algorithms on distributed nodes. Dask also has beautiful diagnostic dashboards that provide users with performance insight.<\/p>\n<p>You can find more information about these tools on the Anaconda website.<\/p>\n<h3>Using AI in Your Organization<\/h3>\n<p>To end his presentation, Oliphant discussed how these tools should be applied within an organization or company. How do you actually go about integrating AI into your technology?<\/p>\n<p>Using machine learning and artificial intelligence in any way requires a process, he said. First, you have to bring your data together. Here, he suggested using visualization tools because the best way to understand your data is to look at it. Next, it is important to do AI brainstorming and consult with people who have real experience in the field. Once you have found the \u201cright\u201d features for your work, it\u2019s time to build and validate your model and repeat that for many models, then publish and manage at some time-scale.<\/p>\n<p>Following such a rigorous approach to machine learning and armed with tools such as Anaconda, we can start making the transition from the AI we have today to a more seamless AI of the future, concluded Oliphant.<\/p>\n<p><strong>Save the date \u2013 March 7th, 2019!<\/strong><br \/>\nNorth Star AI powered by <a href=\"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/about-proekspert\/\">Proekspert<\/a> is coming again and tickets are now available.<br \/>\nMore info: <a href=\"https:\/\/aiconf.tech\/\" target=\"_blank\" rel=\"noopener noreferrer\">aiconf.tech<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>An overview by Travis E. Oliphant. Most tech companies today are struggling to figure out how they can best work with their data. Travis E. Oliphant, the founder of startups Quansight and Anaconda, and the primary developer of NumPy and SciPy packages for Python, gave a talk at North Star AI conference, powered by Proekspert on how Python can be used effectively for machine learning and data \u2013 the heart of ML-driven technology. Oliphant\u2019s connection to Python dates as far back as 1997, when he was working with version 1.4. The very first problem he focused on was actually a data problem, and this led him to understand that in order to do anything with machine learning you have to get the data right. While data seems to be everywhere these days, said Oliphant, the biggest problem is how you gain access to and utilize that information. Applying Python to machine learning problems In Oliphant\u2019s mind, Python is one of the best languages to apply to machine learning problems. During his talk, he gave a thorough overview as to why and presented how he imagines artificial intelligence might develop in the future. Not Artificial but Augmented Intelligence When Oliphant thinks<\/p>\n","protected":false},"author":10,"featured_media":1535,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1534","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bi-data-analytics"],"acf":[],"_links":{"self":[{"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/posts\/1534","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/comments?post=1534"}],"version-history":[{"count":8,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/posts\/1534\/revisions"}],"predecessor-version":[{"id":12045,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/posts\/1534\/revisions\/12045"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/media\/1535"}],"wp:attachment":[{"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/media?parent=1534"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/categories?post=1534"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/clients.triloogia.ee\/proekspert\/wp-new\/wp-json\/wp\/v2\/tags?post=1534"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}