MSK 구성 (미완료)

 

1. 먼저 VPC의 ID부터 확인한다.

 $ aws ec2 describe-vpcs --output table
--------------------------------------------------------------------------------------------------------------------------
|                                                      DescribeVpcs                                                      |
+------------------------------------------------------------------------------------------------------------------------+
||                                                         Vpcs                                                         ||
|+-------------+----------------+------------------+------------+---------------+-------------+-------------------------+|
||  CidrBlock  | DhcpOptionsId  | InstanceTenancy  | IsDefault  |    OwnerId    |    State    |          VpcId          ||
|+-------------+----------------+------------------+------------+---------------+-------------+-------------------------+|
||  10.0.0.0/16|  dopt-4492e12f |  default         |  False     |  369647062832 |  available  |  vpc-0a8664ffdac35e03d  ||
|+-------------+----------------+------------------+------------+---------------+-------------+-------------------------+|
|||                                               CidrBlockAssociationSet                                              |||
||+---------------------------------------------------------------------------------+----------------------------------+||
|||                                  AssociationId                                  |            CidrBlock             |||
||+---------------------------------------------------------------------------------+----------------------------------+||
|||  vpc-cidr-assoc-0aa6c650a34a4f503                                               |  10.0.0.0/16                     |||
||+---------------------------------------------------------------------------------+----------------------------------+||
||||                                                  CidrBlockState                                                  ||||
|||+-------------------------------------------+----------------------------------------------------------------------+|||
||||  State                                    |  associated                                                          ||||
|||+-------------------------------------------+----------------------------------------------------------------------+|||
    auto.create.topics.enable = true
|||                                                        Tags                                                        |||
||+---------------------------------------+----------------------------------------------------------------------------+||
|||                  Key                  |                                   Value                                    |||
||+---------------------------------------+----------------------------------------------------------------------------+||
|||  Name                                 |  oss-bmt-vpc                                                               |||
||+---------------------------------------+----------------------------------------------------------------------------+||
||                                                         Vpcs                                                         ||
|+-------------+----------------+------------------+------------+---------------+-------------+-------------------------+|
||  CidrBlock  | DhcpOptionsId  | InstanceTenancy  | IsDefault  |    OwnerId    |    State    |          VpcId          ||
|+-------------+----------------+------------------+------------+---------------+-------------+-------------------------+|
||  10.0.0.0/16|  dopt-4492e12f |  default         |  False     |  369647062832 |  available  |  vpc-03f8c5fc14943a820  ||
|+-------------+----------------+------------------+------------+---------------+-------------+-------------------------+|
|||                                               CidrBlockAssociationSet                                              |||
||+---------------------------------------------------------------------------------+----------------------------------+||
|||                                  AssociationId                                  |            CidrBlock             |||
||+---------------------------------------------------------------------------------+----------------------------------+||
|||  vpc-cidr-assoc-096c2b5e22a56a206                                               |  10.0.0.0/16                     |||
||+---------------------------------------------------------------------------------+----------------------------------+||
||||                                                  CidrBlockState                                                  ||||
|||+-------------------------------------------+----------------------------------------------------------------------+|||
||||  State                                    |  associated                                                          ||||
|||+-------------------------------------------+----------------------------------------------------------------------+|||
|||                                                        Tags                                                        |||
||+---------------------------------------+----------------------------------------------------------------------------+||
|||                  Key                  |                                   Value                                    |||
||+---------------------------------------+----------------------------------------------------------------------------+||
|||  Name                                 |  urt-bmt-vpc                                                               |||
||+---------------------------------------+----------------------------------------------------------------------------+||

2. vpc-id 확인 후 해당 VPC의 subnet 정보를 확인한다.

$ aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-0a8664ffdac35e03d" --output table | egrep "Name|AvailabilityZone|SubnetId"
||  AvailabilityZone            |  ap-northeast-2a                                                          ||
||  AvailabilityZoneId          |  apne2-az1                                                                ||
||  SubnetId                    |  subnet-0624cd75bfb9cc184                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2d                                                          ||
||  AvailabilityZoneId          |  apne2-az4                                                                ||
||  SubnetId                    |  subnet-0376d55b44d09ed58                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2c                                                          ||
||  AvailabilityZoneId          |  apne2-az3                                                                ||
||  SubnetId                    |  subnet-007ffa198ec558549                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2b                                                          ||
||  AvailabilityZoneId          |  apne2-az2                                                                ||
||  SubnetId                    |  subnet-05862770112c77b57                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2a                                                          ||
||  AvailabilityZoneId          |  apne2-az1                                                                ||
||  SubnetId                    |  subnet-096e203a2cf803925                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2c                                                          ||
||  AvailabilityZoneId          |  apne2-az3                                                                ||
||  SubnetId                    |  subnet-0cd8b79944171649a                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2b                                                          ||
||  AvailabilityZoneId          |  apne2-az2                                                                ||
||  SubnetId                    |  subnet-01f02fbb82988f048                                                 ||
|||  Key                          |  Name                                                                  |||
||  AvailabilityZone            |  ap-northeast-2d                                                          ||
||  AvailabilityZoneId          |  apne2-az4                                                                ||
||  SubnetId                    |  subnet-00dd6c87cfd3b43ed                                                 ||
|||  Key                          |  Name                                                                  |||

 3. msk config 파일 작성

아래 내용으로 msk cluster config 내용을 생성한다.

 

cluster_config.txt

    auto.create.topics.enable = true
    delete.topic.enable = true
    log.retention.hours = 8

 

4. kafka cluster config를 생성한다.