Chapter 8: Big data

The basic idea behind big data is that the more good data is available, the better able we are to exploit it. In practice, we use machine learning techniques to do the exploitation - they are able to build models that can fit or describe the characteristsics of the data that they have been trained with.


8.3.1 Data storage and access

Loading items from a subdirectory is something we need to do often when we deal with big data... because it is almost always stored in separate files in such subdirectories. The code given in the book for doing this is as follows;

dir_name='~/Documents/MATLAB/MyFiles/';
sub_d=dir(dir_name); 
for fi=2:length(sub_d)
    % 2 because the first 2 files are current 
    %and parent dir , in unix 
    fname=sub_d(fi).name;
    % [wave,fs]=audioread([dir_name ,fname]);
    fprintf('%s\n',[dir_name ,fname])
end

WORK IN PROGRESS - please revisit later for additional information to come in this section