Category Archives: big data

big data, metric, machine learning, large datasets, internet, massive data

The reign and modern challenges of the Message Passing Interface (MPI): A discussion with Dr. Torsten Hoefler.

Posted on by
Reply

A few years ago, while I was a graduate student in Greece, I was preparing  slides for my talk at the SIAM Parallel Processing 2012 conference. While showing my slides to one of my colleagues, one of his comments was: “All good, but why do you guys doing numerical linear algebra and parallel computing always use the Message Passing Interface to communicate between the processors?”. Having read* the book review of Beresford Parlett in [1], I did have the wit to imitate Marvin Minsky and reply “Is there any other way?”. Nowadays, this question is even more interesting, and my answer would certainly be longer (perhaps too long!). Execution of programs in distributed computing environments requires communication between the processors. It is then natural to consider by what protocols and guidelines should the processors communicate with each other? This is the question to which the Message Passing Interface (MPI) has been the answer for more than 25 years.

Continue reading →

How cloud computing could improve your research.

Posted on by
Reply

Processing large datasets or working with huge data is a usual stage in almost any kind of research. The problem appears when processing these data requires long processing time on any kind of personal laptop.  In order to solve that issue, we have at our hands the possibility of using any major compute cloud provider in order to accelerate that processing stage wasting minimal time and resources. Cloud computing allows us with a few clicks to build a computer cluster, process our data, get the results and destroy that cluster for only a few dollars.

Continue reading →

Exascale computing: Why we need it and some of the challenges to conquer it.

Posted on by
1

To introduce you to Exascale computing, as well as its challenges, we interviewed the distinguished Professor Jack Dongarra (University of Tennessee), an internationally renowned expert in high-performance computing and the leading scientist behind the TOP500 (http://www.top500.org/), a list which ranks supercomputers according to their performance.

Continue reading →

A Brief Survey on Software Recommendation Tools

Posted on by
Reply

Software engineering recommendation systems assist developers so that they can almost automatically find: 1) code snippets that match their programs, 2) appropriate APIs and libraries, 3) bug fixes, and 4) code changes. These systems are particularly important because they can help developers to handle large amounts of information and write stable programs. In this post, we briefly categorize existing recommendation approaches and tools.

Continue reading →

R software and tools for everyday use

Posted on by
Reply

Long, long time ago … I started with Octave and Matlab.They were amazing and allowed me to solve a lot of interesting problems in my research. I loved the command window of Octave, but I needed the productivity an IDE gives when developing complex calculations. None of the available IDE’s for Octave were not as powerful as the Matlab IDE. The problem was that Matlab was not GNU and buying a license was very expensive. Then, I found R and I realized that none Octave neither Matlab were the tool I needed for my research. I needed advanced project and file management through repositories, fast data manipulation, an easy way to export my calculations, a creative way of authoring reports and a powerful IDE that let me access my beloved command window. Now R gives me all I need and is an important part of my everyday toolbox. For those who does not known R, I must say that R is a well known programming language that is widely used on mathematics, economy, biology… Its main benefits includes the ability to work easily with statistics and data manipulation. R is very popular on academics and research, is GNU, very powerful and have a lot of packages that allows do magical things in a few clicks or with a few commands.

Continue reading →