哪一个是最好的CSV或JSON才能导入大数据(PHP)[关闭]

哪一个是最好的CSV或JSON才能导入大数据(PHP)[关闭]

问题描述:

I'm trying to get a big number of data (about 3M rows) and I have only two options to do that.

  1. Call an API then recover the 3M JSON objects.
  2. Import a CSV file containing the 3M rows.

I didn't test any of these solutions yet to tell which one is best in terms of speed.

我正在尝试获取大量数据(约3M行),我只有两个选项可做 那个。 p>

  1. 调用API然后恢复3M JSON对象。 li>
  2. 导入包含3M行的CSV文件。 li> ol>

    我还没有测试任何这些解决方案,以确定速度方面哪一个最佳。 p> div>

If you want to retrieve simple data as lists or rows with some columns the option #2 is the good one, you can read below a set of advantages and disadvantages:

Pros

  • Less bandwidth needed because json needs more syntax characters to keep the format while csv is as simple as use a character separator
  • Process data is faster because only needs to split by the separator character while JSON needs to interpret the syntax
  • Big data technology as Hadoop have an integrated parse for CSV format while needs an specific function for parse Json (for example using Hive language).

Cons

  • Unstructured data and more difficult to be read by humans
  • You have to take care with separator character cannot appear in data fields.

If the data will contains complex data as tupla, arrays and structures JSON is better because:

  • Keeps a clear and structured format
  • Doesn't repeat data to reference it because one label could contain multiple data.