Currently, only data for one-way trips are counted.
mysql
statistical tasks
ID
trip_type
statistics_status
data_status
enable_status
cap_channel
Crawl channels
create_time
last_update_time
Task results
ID
task_id
trip_type
cap_channel
carrier
air_line
valid_fare_days
valid_routing_number
data_status
enable_status
create_time
last_update_time
process
1. Obtain the incoming ticket capture channel and obtain a list of all related capture strategies based on the ticket capture channel.
If not transmitted, all channels will be checked by default.
2. According to the queried crawling strategy list, obtain the whitelist list in the strategy list respectively.
Establish a ticket capture channel - airline - route relationship list
3. According to the airline-route relationship list, the date is represented by * wildcard. According to the key rules, all the main keys under a ticket capture channel are spliced together in batches at one time and stored in the set.
4. Use mget to set the main key to mget all keys in redis at once to obtain a return list.
5. Filter the returned list and group it by route
There are several pieces of data in the group that represent how many days there are tickets for this route.
The sum of the routing sizes of each piece of data in the group represents the effective number of trips
6. Generate statistical result data under this strategy and store it in the database
If there are multiple ticket capture channels, repeat steps 2 to 6 to store the statistical results under each strategy.