MindMap Gallery Effective fare statistics
This is a mind map about effective fare statistics. The main contents include: process, database, and execution methods. Welcome to like and collect~
Edited at 2024-03-08 13:58:29This is a mind map about bacteria, and its main contents include: overview, morphology, types, structure, reproduction, distribution, application, and expansion. The summary is comprehensive and meticulous, suitable as review materials.
This is a mind map about plant asexual reproduction, and its main contents include: concept, spore reproduction, vegetative reproduction, tissue culture, and buds. The summary is comprehensive and meticulous, suitable as review materials.
This is a mind map about the reproductive development of animals, and its main contents include: insects, frogs, birds, sexual reproduction, and asexual reproduction. The summary is comprehensive and meticulous, suitable as review materials.
This is a mind map about bacteria, and its main contents include: overview, morphology, types, structure, reproduction, distribution, application, and expansion. The summary is comprehensive and meticulous, suitable as review materials.
This is a mind map about plant asexual reproduction, and its main contents include: concept, spore reproduction, vegetative reproduction, tissue culture, and buds. The summary is comprehensive and meticulous, suitable as review materials.
This is a mind map about the reproductive development of animals, and its main contents include: insects, frogs, birds, sexual reproduction, and asexual reproduction. The summary is comprehensive and meticulous, suitable as review materials.
Effective fare statistics
Implementation modalities
Select the ticket capture channel on the page and click Add Statistics Task to start execution.
database
redis
key
Prefix: itinerary type~departure airport~arrival airport~departure time~airline company~
fare:info:1~CEI~DMK~2024-03-11~FD~
Currently, only data for one-way trips are counted.
mysql
statistical tasks
ID
trip_type
statistics_status
data_status
enable_status
cap_channel
Crawl channels
create_time
last_update_time
Task results
ID
task_id
trip_type
cap_channel
carrier
air_line
valid_fare_days
valid_routing_number
data_status
enable_status
create_time
last_update_time
process
1. Obtain the incoming ticket capture channel and obtain a list of all related capture strategies based on the ticket capture channel.
If not transmitted, all channels will be checked by default.
2. According to the queried crawling strategy list, obtain the whitelist list in the strategy list respectively.
Establish a ticket capture channel - airline - route relationship list
3. According to the airline-route relationship list, the date is represented by * wildcard. According to the key rules, all the main keys under a ticket capture channel are spliced together in batches at one time and stored in the set.
4. Use mget to set the main key to mget all keys in redis at once to obtain a return list.
5. Filter the returned list and group it by route
There are several pieces of data in the group that represent how many days there are tickets for this route.
The sum of the routing sizes of each piece of data in the group represents the effective number of trips
6. Generate statistical result data under this strategy and store it in the database
If there are multiple ticket capture channels, repeat steps 2 to 6 to store the statistical results under each strategy.