Skip to content

Latest commit

 

History

History
112 lines (89 loc) · 3.61 KB

README.md

File metadata and controls

112 lines (89 loc) · 3.61 KB
Raw

Huawei Cloud MRS example code

The Huawei Cloud MRS example code include HBase, HDFS, Hive, Kafka, Mapreduce, HetuEngine, Spark, Elasticsearch, ClickHouse etc. You can get started in minutes using Maven.

Version Mapping:

MRS_3.2.0.1 Components mapping:

Component\MRS version MRS 3.2.0.1
Flink 1.15.0
Hive 3.1.0
Tez 0.9.2
Spark 3.1.1
CarbonData 2.2.0
Hudi 0.11.0
Hadoop 3.3.1
HBase 2.2.3
ZooKeeper 3.6.3
Hue 4.7.0
Oozie 5.1.0
Flume 1.9.0
Kafka 2.4.0
Ranger 2.0.0
Solr 8.4.0
Phoenix 5.0.0
Elasticsearch 7.10.2
ClickHouse 22.3.2.2
IoTDB 0.14.0
Redis 6.0.12
HetuEngine 1.2.0

Quick Links:

Getting Started

For more details, please ref to MRS's Deveployer Guide.

Requirements

To run the examples required:

  • Java 1.8+
  • Maven 3.0+

Specify the Maven Repository

Add the following open source mirror repository address to mirrors in the settings.xml configuration file.

<mirror>
    <id>repo2</id>
    <mirrorOf>central</mirrorOf>
    <url>https://repo1.maven.org/maven2/</url>
</mirror>

Add the following mirror repository address to profiles in the settings.xml configuration file.

<profile>
    <id>huaweicloudsdk</id>
    <repositories>
        <repository>
            <id>huaweicloudsdk</id>
            <url>https://repo.huaweicloud.com/repository/maven/huaweicloudsdk/</url>
            <releases><enabled>true</enabled></releases>
            <snapshots><enabled>true</enabled></snapshots>
        </repository>
    </repositories>
</profile>
<profile>
    <id>JDK1.8</id>
    <activation>
        <activeByDefault>true</activeByDefault>
        <jdk>1.8</jdk>
    </activation>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <maven.compiler.encoding>UTF-8</maven.compiler.encoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
    </properties>
</profile>

Add the following mirror repository address to the activeProfiles node in the settings.xml file.

<activeProfile>huaweicloudsdk</activeProfile>

Building From Source

Once you check out the code from GitHub, you can build it using maven for every child project, eg:

cd src\hdfs-example-security
mvn clean install

License

Apache License 2.0