大数据之 Hadoop-11-2-Sqoop 实战

在前边 大数据之 Hadoop-11-Sqoop 学习了基础知识和姜MySQL数据库的表导入到HDFS，这一节，学习将Hdfs数据导出到MySQL表。

一、将HDFS数据导出到MySQL

下边将 /sqoop/mysql/stocks/part-m-0000x 中的数据导出到MySQL表中。

1、新建表

2、执行导出命令

执行以下命令，将 HDFS 文件系统 /sqoop/mysql/stocks/part-m-0000x 文件中的内容导出到表 stock_prices_test 中：

sqoop export \
--connect 'jdbc:mysql://98.12.13.15:3307/demodb?characterEncoding=UTF-8' \
--username root \
--password Quant88 \
--table stock_prices_test \
--export-dir /sqoop/mysql/stocks/*

/sqoop/mysql/stocks/* 表示将 HDFS 中 /sqoop/mysql/stocks/ 目录下的所有文件一起导出到 stock_prices_test 表中，如果只需要导入一个文件，则为： /sqoop/mysql/stocks/part-m-00001。

打印结果：

2021-09-12 07:58:09,895 INFO mapreduce.Job: Running job: job_1631403489943_0001
2021-09-12 07:58:42,138 INFO mapreduce.Job: Job job_1631403489943_0001 running in uber mode : false
2021-09-12 07:58:42,141 INFO mapreduce.Job:  map 0% reduce 0%
2021-09-12 07:59:25,520 INFO mapreduce.Job:  map 33% reduce 0%
2021-09-12 07:59:26,573 INFO mapreduce.Job:  map 100% reduce 0%
2021-09-12 07:59:28,620 INFO mapreduce.Job: Job job_1631403489943_0001 completed successfully
2021-09-12 07:59:28,873 INFO mapreduce.Job: Counters: 33
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=679266
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=15275
        HDFS: Number of bytes written=0
        HDFS: Number of read operations=21
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=0
    Job Counters 
        Launched map tasks=3
        Other local map tasks=1
        Data-local map tasks=2
        Total time spent by all maps in occupied slots (ms)=123156
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=123156
        Total vcore-milliseconds taken by all map tasks=123156
        Total megabyte-milliseconds taken by all map tasks=126111744
    Map-Reduce Framework
        Map input records=77
        Map output records=77
        Input split bytes=606
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=1481
        CPU time spent (ms)=5950
        Physical memory (bytes) snapshot=340766720
        Virtual memory (bytes) snapshot=8209305600
        Total committed heap usage (bytes)=91422720
        Peak Map Physical memory (bytes)=114606080
        Peak Map Virtual memory (bytes)=2736435200
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=0
2021-09-12 07:59:28,908 INFO mapreduce.ExportJobBase: Transferred 14.917 KB in 92.4588 seconds (165.2088 bytes/sec)
2021-09-12 07:59:28,920 INFO mapreduce.ExportJobBase: Exported 77 records.
[hadoop@centos01 sbin]$ 

查看表：

可以看到表已经导入成功了。

为者常成，行者常至