php - Getting Data From a Huge JSON File and Transfer it to CSV or SQL

Question

Originally I have JSON file containing 1Million+ books Information and Size is approx 3.1 GB.

I want to transfer this data to some other format to use it more comfortably in sql or csv or other.

Is there any way to do it in simple way.

I know php but 3.1GB is so huge for my system to handle.

suggest any other language(with code to do the same i only understand php) or application that can do the same.

Answer 1

used halaxa/json-machine.

Usage in case of iteration over JSON is the same as in case of json_decode, but it will not hit memory limit no matter how big your file is. No need to implement anything, just your foreach.

Example:

$users = JsonMachineJsonMachine::fromFile('500MB-users.json');

foreach ($users as $id => $user) {
    // process $user as usual
}

See github readme for more details.

One alternative here is to use the salsify/jsonstreamingparser

You need to create your own Listener.

$testfile = '/path/to/file.json';
$listener = new MyListener();
$stream = fopen($testfile, 'r');
try {
    $parser = new JsonStreamingParserParser($stream, $listener);
    $parser->parse();
    fclose($stream);
} catch (Exception $e) {
    fclose($stream);
    throw $e;
}